-- Logs begin at Fri 2022-09-23 14:14:44 UTC, end at Fri 2022-09-23 15:39:56 UTC. -- Sep 23 14:57:30.680791 dv-cinder-855307-2-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Sep 23 14:57:30.690869 dv-cinder-855307-2-iscsi devstack@keystone.service[81981]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Sep 23 14:57:30.693891 dv-cinder-855307-2-iscsi devstack@keystone.service[81981]: *** Starting uWSGI 2.0.18-debian (64bit) on [Fri Sep 23 14:57:30 2022] *** Sep 23 14:57:30.693891 dv-cinder-855307-2-iscsi devstack@keystone.service[81981]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Sep 23 14:57:30.693891 dv-cinder-855307-2-iscsi devstack@keystone.service[81981]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Sep 23 14:57:30.693891 dv-cinder-855307-2-iscsi devstack@keystone.service[81981]: nodename: dv-cinder-855307-2-iscsi Sep 23 14:57:30.693891 dv-cinder-855307-2-iscsi devstack@keystone.service[81981]: machine: x86_64 Sep 23 14:57:30.693891 dv-cinder-855307-2-iscsi devstack@keystone.service[81981]: clock source: unix Sep 23 14:57:30.693891 dv-cinder-855307-2-iscsi devstack@keystone.service[81981]: pcre jit disabled Sep 23 14:57:30.693891 dv-cinder-855307-2-iscsi devstack@keystone.service[81981]: detected number of CPU cores: 4 Sep 23 14:57:30.693891 dv-cinder-855307-2-iscsi devstack@keystone.service[81981]: current working directory: / Sep 23 14:57:30.694823 dv-cinder-855307-2-iscsi devstack@keystone.service[81981]: detected binary path: /usr/bin/uwsgi-core Sep 23 14:57:30.694823 dv-cinder-855307-2-iscsi devstack@keystone.service[81981]: your processes number limit is 47680 Sep 23 14:57:30.694823 dv-cinder-855307-2-iscsi devstack@keystone.service[81981]: your memory page size is 4096 bytes Sep 23 14:57:30.694823 dv-cinder-855307-2-iscsi devstack@keystone.service[81981]: detected max file descriptor number: 2048 Sep 23 14:57:30.694823 dv-cinder-855307-2-iscsi devstack@keystone.service[81981]: lock engine: pthread robust mutexes Sep 23 14:57:30.694823 dv-cinder-855307-2-iscsi devstack@keystone.service[81981]: thunder lock: enabled Sep 23 14:57:30.694823 dv-cinder-855307-2-iscsi devstack@keystone.service[81981]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Sep 23 14:57:30.694823 dv-cinder-855307-2-iscsi devstack@keystone.service[81981]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Sep 23 14:57:30.735947 dv-cinder-855307-2-iscsi devstack@keystone.service[81981]: Python main interpreter initialized at 0x556a386dc9c0 Sep 23 14:57:30.735947 dv-cinder-855307-2-iscsi devstack@keystone.service[81981]: python threads support enabled Sep 23 14:57:30.735947 dv-cinder-855307-2-iscsi devstack@keystone.service[81981]: your server socket listen backlog is limited to 100 connections Sep 23 14:57:30.735947 dv-cinder-855307-2-iscsi devstack@keystone.service[81981]: your mercy for graceful operations on workers is 90 seconds Sep 23 14:57:30.736381 dv-cinder-855307-2-iscsi devstack@keystone.service[81981]: mapped 403077 bytes (393 KB) for 2 cores Sep 23 14:57:30.736457 dv-cinder-855307-2-iscsi devstack@keystone.service[81981]: *** Operational MODE: preforking *** Sep 23 14:57:30.736702 dv-cinder-855307-2-iscsi devstack@keystone.service[81981]: *** uWSGI is running in multiple interpreter mode *** Sep 23 14:57:30.736702 dv-cinder-855307-2-iscsi devstack@keystone.service[81981]: spawned uWSGI master process (pid: 81981) Sep 23 14:57:30.736914 dv-cinder-855307-2-iscsi systemd[1]: Started Devstack devstack@keystone.service. Sep 23 14:57:30.738946 dv-cinder-855307-2-iscsi devstack@keystone.service[81981]: spawned uWSGI worker 1 (pid: 81993, cores: 1) Sep 23 14:57:30.738946 dv-cinder-855307-2-iscsi devstack@keystone.service[81981]: spawned uWSGI worker 2 (pid: 81994, cores: 1) Sep 23 14:57:30.738946 dv-cinder-855307-2-iscsi devstack@keystone.service[81981]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Sep 23 14:57:35.176825 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 23 14:57:35.176825 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: from cryptography.utils import int_from_bytes Sep 23 14:57:35.177608 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 23 14:57:35.177608 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: from cryptography.utils import int_from_bytes Sep 23 14:57:35.178050 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 23 14:57:35.178050 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: from cryptography.utils import int_from_bytes Sep 23 14:57:35.178169 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 23 14:57:35.178169 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: from cryptography.utils import int_from_bytes Sep 23 14:57:35.238822 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 23 14:57:35.239024 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 23 14:57:35.239208 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] command line args: [] {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 23 14:57:35.239486 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 23 14:57:35.239675 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] ================================================================================ {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 23 14:57:35.239759 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 23 14:57:35.239875 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 23 14:57:35.239971 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] admin_token = **** {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 14:57:35.240114 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] command line args: [] {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 23 14:57:35.240194 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] config_dir = [] {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 14:57:35.240362 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 23 14:57:35.240442 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 14:57:35.240543 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] ================================================================================ {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 23 14:57:35.240622 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] config_source = [] {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 14:57:35.240738 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] debug = True {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 14:57:35.240822 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] admin_token = **** {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 14:57:35.240944 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 14:57:35.241203 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] config_dir = [] {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 14:57:35.241203 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 14:57:35.241317 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 14:57:35.241381 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] config_source = [] {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 14:57:35.241527 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 14:57:35.241605 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] debug = True {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 14:57:35.241752 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 14:57:35.241831 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 14:57:35.241994 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 14:57:35.242127 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 14:57:35.242228 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 14:57:35.242330 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] list_limit = None {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 14:57:35.242426 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] log_config_append = None {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 14:57:35.242528 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 14:57:35.242624 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 14:57:35.242728 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 14:57:35.242826 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] log_dir = None {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 14:57:35.242928 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] list_limit = None {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 14:57:35.243031 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] log_file = None {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 14:57:35.243129 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] log_config_append = None {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 14:57:35.243248 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 14:57:35.243333 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 14:57:35.243554 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 14:57:35.243687 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] log_dir = None {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 14:57:35.243687 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] log_file = None {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 14:57:35.243924 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 14:57:35.244054 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 14:57:35.244189 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 14:57:35.244189 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 14:57:35.244398 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 14:57:35.244476 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 14:57:35.244595 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 14:57:35.244679 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 14:57:35.244753 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 14:57:35.244874 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 14:57:35.244979 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 14:57:35.245058 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 14:57:35.245125 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 14:57:35.245277 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 14:57:35.245363 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 14:57:35.245532 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 14:57:35.245618 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 14:57:35.245755 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 14:57:35.245832 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 14:57:35.245995 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 14:57:35.246081 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 14:57:35.246233 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 14:57:35.246341 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 14:57:35.246472 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 14:57:35.246557 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 14:57:35.246761 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 14:57:35.247014 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 14:57:35.247091 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.55/identity {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 14:57:35.247211 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] publish_errors = False {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 14:57:35.247298 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 14:57:35.247429 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 14:57:35.247667 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 14:57:35.247787 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.55/identity {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 14:57:35.247941 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 14:57:35.248040 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] publish_errors = False {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 14:57:35.251154 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 14:57:35.251272 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 14:57:35.251398 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 14:57:35.251484 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 14:57:35.251573 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] standard_threads = False {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 14:57:35.251654 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 14:57:35.251732 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 14:57:35.251809 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 14:57:35.251891 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 14:57:35.251964 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 14:57:35.252068 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 14:57:35.252148 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] standard_threads = False {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 14:57:35.252220 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] use_journal = False {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 14:57:35.252290 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 14:57:35.252367 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] use_json = False {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 14:57:35.252367 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] use_stderr = False {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 14:57:35.252507 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 14:57:35.252579 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] use_syslog = False {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 14:57:35.252654 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 14:57:35.252654 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] use_journal = False {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 14:57:35.252791 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 14:57:35.252791 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.252917 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] use_json = False {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 14:57:35.252987 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.253057 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] use_stderr = False {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 14:57:35.253057 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] use_syslog = False {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 14:57:35.253268 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.253347 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 14:57:35.253424 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.253424 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.253546 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.253623 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.253705 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.253777 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.253853 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.253924 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.253999 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.254070 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.254147 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.254224 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.254295 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.254370 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.254446 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.254518 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.254595 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.254666 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.254736 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.254811 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.254888 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.254970 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.255042 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.255113 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.255196 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.255279 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.255351 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.255429 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.255502 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.255572 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.255641 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.255706 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.255777 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.255857 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.255934 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.256041 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.256115 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.256186 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.256255 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.256337 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.256415 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.256487 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.256557 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.256627 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.256696 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.256840 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.256914 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.256984 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.257054 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.257124 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.257239 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.257332 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.257477 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.257555 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.257714 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.257793 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.257950 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.258034 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.258188 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.258271 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.258441 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.258517 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.258685 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.258775 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.258918 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.259001 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.259684 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.259769 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.259927 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.260042 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.260373 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.260497 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.260595 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.260697 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.260794 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.260900 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.260999 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.261096 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.261194 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.261329 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.261468 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.261547 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.261666 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.261742 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.261858 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.261936 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.262017 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.262097 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.262257 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.262388 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.262470 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.262619 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.262737 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.262827 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.262950 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.263096 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.263177 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.263339 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.263419 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.263576 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.263664 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.263817 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.263894 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.264080 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.264164 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.264335 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.264429 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.264571 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.264650 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.264813 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.264891 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.265055 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.265133 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.265292 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.265368 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.265527 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.265617 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.265764 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.265862 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.265999 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.266079 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.266250 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.266329 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.266489 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.266575 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.266727 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.266810 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.266965 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.267052 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.267229 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.267320 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.267469 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.267548 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.267705 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.267783 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.268118 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.268265 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.268354 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.268427 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.268499 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.268569 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.268685 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.268848 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.268923 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.269032 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.269156 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.269281 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.269403 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.269522 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.269643 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.269762 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.269897 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.270002 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.270139 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.270256 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.270378 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.270498 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.270619 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.270739 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.270860 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.270983 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.271100 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.271223 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.271372 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.271476 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.271624 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.271720 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.271880 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.271960 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.272235 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.272316 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.272483 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.272562 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.272734 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.272815 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.272977 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.273056 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.273219 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.273382 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.273516 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.273602 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.273676 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.273836 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.273921 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.274078 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.274166 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.274325 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.274404 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.274566 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.274645 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.275369 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.275526 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.275636 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.275751 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.275874 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.275938 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.276151 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.279520 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.279520 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.279697 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.279770 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.279841 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.279912 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.280023 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.280102 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] auth.external = None {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.280173 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.280244 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.280314 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.280385 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.280472 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.280662 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.280758 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.280840 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] auth.password = None {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.280913 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.280986 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] auth.token = None {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.281057 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.281129 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.281209 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.281282 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.281367 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] auth.external = None {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.281449 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.281522 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.281848 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.281933 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.282007 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.282007 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.282144 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.282221 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.282292 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] auth.password = None {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.282371 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] credential.caching = True {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.282444 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] auth.token = None {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.282515 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.282591 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.282663 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.282734 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.282813 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.282892 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.282964 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.283041 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.283114 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.283185 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.283264 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.283343 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.283416 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.283487 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.283558 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.283648 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] credential.caching = True {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.283722 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.283793 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.283865 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.283936 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.283936 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.284120 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.284196 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.284268 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.284339 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.284410 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.284487 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.284654 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.284731 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.284803 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.284876 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.284947 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] federation.caching = True {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.285019 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.285099 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.285172 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.285243 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.285315 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.285386 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.285386 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.285512 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.285574 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.285634 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.285722 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.285869 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.285999 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.286135 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.286216 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.286378 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] federation.caching = True {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.286458 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.286622 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.286702 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.286874 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.286955 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.287125 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.287205 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] identity.caching = True {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.287362 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.287441 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.287614 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.287695 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.287860 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.287938 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.288127 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.288217 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.288373 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.288453 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.288608 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.288687 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.288852 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.288935 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.289105 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.289184 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.289347 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] identity.caching = True {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.289425 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.289584 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.289661 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.289821 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.289901 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.290057 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.290143 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.290333 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.290411 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.290592 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.290671 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.290825 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.290904 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.291068 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.291147 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.291365 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.291498 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.291701 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.291824 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.291947 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.292162 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.292292 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.292421 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.292548 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.292687 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.292814 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.292949 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.292949 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.293187 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.293313 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.293441 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.293567 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.293701 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.293828 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.293955 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.294081 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.294207 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.294343 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.294480 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.294607 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.294732 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.294858 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.294983 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.295109 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.295242 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.295369 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.295497 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.295497 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.295735 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.295735 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.295993 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.295993 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.296246 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.296246 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.296479 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.296479 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.296703 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.296703 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.296833 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.296904 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.296976 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.297048 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.297128 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.297202 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.297274 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.297340 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.297497 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.297708 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.297799 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.297952 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.298030 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.298185 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.298271 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.298427 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.298514 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.298670 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.298750 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.298918 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.298995 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.299159 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.299245 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.299402 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.299481 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.299641 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] ldap.user = None {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.299719 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.299889 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.300061 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.300185 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.300304 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.300427 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.300543 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.300684 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.300798 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.300939 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.301052 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.301195 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.301339 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.301422 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.301584 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.301665 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.301828 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.301908 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.302067 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] ldap.user = None {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.302145 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.302314 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.302395 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.302576 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.302660 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.302820 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.302899 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.303117 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.303207 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.303321 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.303400 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.303573 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.303651 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.303813 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.303894 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.304074 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.304160 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.304318 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.304400 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.304559 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.304640 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.304801 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.304879 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.305035 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.305123 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.305276 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.305355 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.305528 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.305606 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.305781 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.305870 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.306679 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.307042 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.307132 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.307285 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.307364 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.307530 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.307610 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.307778 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.307856 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.308046 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.308130 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.308293 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.308415 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.308519 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.308639 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.308756 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.308869 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.309000 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.309115 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.309232 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.309348 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.309466 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] resource.caching = True {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.309575 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.309700 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.309823 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.309942 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.310059 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.310183 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.310299 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.310426 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.310543 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.310663 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.310790 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.310886 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.310961 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.311100 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.311184 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.311347 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.311427 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.311584 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.311663 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.311824 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.311905 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] role.caching = True {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.312092 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] resource.caching = True {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.312174 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] role.driver = sql {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.312333 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.312411 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.312589 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.312670 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.312845 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.312939 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.313089 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.313297 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.313374 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.313438 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.313585 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.313717 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.313847 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.313975 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.314110 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.314239 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.314366 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.314495 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.314622 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.314749 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] role.caching = True {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.314885 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.315015 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] role.driver = sql {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.315151 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.315279 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.315407 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.315535 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.315672 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.315800 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.315800 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.316088 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.316212 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.316340 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.316478 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.316563 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.316728 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.316866 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.316994 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.317129 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.317257 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.317383 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.317509 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.317637 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.317764 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.317899 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.318028 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.318162 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.318289 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.318416 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.318552 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.318689 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.318689 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.318929 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.319056 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.319182 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.319308 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.319442 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.319527 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.319596 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.319659 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.319719 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.319778 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.319845 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.320035 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.320111 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.320278 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.320347 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.320521 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] token.caching = True {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.320602 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.320765 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.320833 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.320998 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.321067 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.321248 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.321333 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.321543 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.322283 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.322406 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.322757 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.322893 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.323029 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.323153 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.323282 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.323417 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.323507 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.323684 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.323776 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.323852 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.323924 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.324143 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.331470 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] token.caching = True {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.331598 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.331693 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.331772 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.331846 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.331919 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.332012 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.332085 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.332085 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.332228 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.332292 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.332352 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.332412 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] database.connection = **** {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.332472 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.332531 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.332599 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.332599 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.332707 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.332707 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.332812 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.332812 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.332924 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.332985 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.333045 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.333104 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.333164 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.333223 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.333290 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.333290 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.333397 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.333457 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.333517 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.333584 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.333653 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.333792 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.333864 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.333926 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.333986 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] database.connection = **** {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.334045 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.334120 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.334190 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.334251 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.334311 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.334377 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.334438 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.334498 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.334565 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.334627 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.334687 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.334746 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.334805 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81993) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 23 14:57:35.334865 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.334865 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.334865 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.334865 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.334865 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.334865 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.334865 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.334865 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.334865 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.335341 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.335341 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.335341 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 14:57:35.335341 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81994) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 23 14:57:35.360168 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81993) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 23 14:57:35.360401 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81993) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 23 14:57:35.360632 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81993) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 23 14:57:35.362649 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81994) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 23 14:57:35.362887 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81994) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 23 14:57:35.363125 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81994) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 23 14:57:35.370208 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81993) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 23 14:57:35.373110 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81994) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 23 14:57:35.400386 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81993) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 23 14:57:35.400606 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81993) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 23 14:57:35.401760 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81994) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 23 14:57:35.402037 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81994) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 23 14:57:35.469378 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81993) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 23 14:57:35.469937 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81994) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 23 14:57:35.520403 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81993) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 23 14:57:35.520633 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81993) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 23 14:57:35.522170 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81994) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 23 14:57:35.522406 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81994) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 23 14:57:35.680163 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81993) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 23 14:57:35.680163 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81993) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 23 14:57:35.680542 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81993) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 23 14:57:35.680645 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81993) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 23 14:57:35.680789 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81993) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 23 14:57:35.681591 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81994) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 23 14:57:35.681742 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81993) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 23 14:57:35.681871 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81994) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 23 14:57:35.682052 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81994) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 23 14:57:35.682375 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81994) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 23 14:57:35.682606 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81994) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 23 14:57:35.682828 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81994) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 23 14:57:35.704950 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81993) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 23 14:57:35.705168 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81994) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 23 14:57:35.705284 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81993) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 23 14:57:35.705396 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81993) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 23 14:57:35.705481 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81994) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 23 14:57:35.705626 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81993) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 23 14:57:35.705718 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81994) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 23 14:57:35.705849 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81993) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 23 14:57:35.705932 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81994) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 23 14:57:35.706068 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81994) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 23 14:57:35.706149 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81993) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 23 14:57:35.706287 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81993) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 23 14:57:35.706377 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81994) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 23 14:57:35.706503 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81994) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 23 14:57:35.717885 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81993) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 23 14:57:35.725075 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81994) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 23 14:57:35.731129 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81993) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 23 14:57:35.731370 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81993) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 23 14:57:35.731605 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81993) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 23 14:57:35.731835 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81993) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 23 14:57:35.732086 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81993) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 23 14:57:35.732312 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81993) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 23 14:57:35.732534 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81993) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 23 14:57:35.736750 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81994) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 23 14:57:35.736983 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81994) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 23 14:57:35.737213 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81994) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 23 14:57:35.737441 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81994) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 23 14:57:35.737664 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81994) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 23 14:57:35.737883 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81994) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 23 14:57:35.738101 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81994) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 23 14:57:35.755745 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81994) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 14:57:35.755896 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81993) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 14:57:35.756118 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81993) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 23 14:57:35.756224 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81994) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 23 14:57:35.756306 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81993) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 14:57:35.756380 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81994) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 14:57:35.756498 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81993) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 23 14:57:35.756582 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81994) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 23 14:57:35.756776 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81994) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 14:57:35.756878 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81993) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 14:57:35.757038 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81993) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 14:57:35.757126 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81994) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 14:57:35.757297 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81993) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 14:57:35.757379 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81994) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 14:57:35.757557 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81993) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 14:57:35.757643 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81994) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 14:57:35.767246 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81993) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 14:57:35.767518 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81994) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 14:57:35.767624 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81993) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 14:57:35.767790 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81994) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 14:57:35.767899 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81993) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 14:57:35.768100 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81993) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 14:57:35.768187 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81994) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 14:57:35.768388 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81994) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 14:57:35.775120 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81993) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 23 14:57:35.775480 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81994) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 23 14:57:35.778033 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81993) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 23 14:57:35.778320 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81993) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 14:57:35.778607 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81993) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 14:57:35.778782 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81994) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 23 14:57:35.778883 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81993) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 14:57:35.779081 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81994) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 14:57:35.779173 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81993) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 14:57:35.779369 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81994) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 14:57:35.779453 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81993) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 14:57:35.780213 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81994) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 14:57:35.780332 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81993) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 14:57:35.780530 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81994) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 14:57:35.780615 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81993) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 14:57:35.781240 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81994) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 14:57:35.781327 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81993) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 14:57:35.781415 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81994) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 14:57:35.781492 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81993) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 14:57:35.781492 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81993) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 14:57:35.781980 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81994) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 14:57:35.781980 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81994) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 14:57:35.781980 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81994) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 14:57:35.781980 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81994) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 14:57:35.804171 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81993) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 14:57:35.805696 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81993) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 23 14:57:35.806553 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81993) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 14:57:35.806654 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81994) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 14:57:35.808282 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81994) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 23 14:57:35.808569 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81994) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 14:57:35.811334 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81993) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 23 14:57:35.811621 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81993) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 14:57:35.811905 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81993) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 14:57:35.813782 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81994) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 23 14:57:35.813782 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81994) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 14:57:35.813782 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81994) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 14:57:35.818284 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81993) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 23 14:57:35.818575 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81993) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 14:57:35.820303 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81994) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 23 14:57:35.820592 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81994) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 14:57:35.824495 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81993) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 23 14:57:35.824849 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81993) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 14:57:35.825086 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81993) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 14:57:35.825374 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81993) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 14:57:35.825662 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81993) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 14:57:35.825938 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81993) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 14:57:35.826209 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81993) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 14:57:35.826482 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81993) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 14:57:35.826609 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81994) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 23 14:57:35.826888 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81994) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 14:57:35.827240 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81994) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 14:57:35.827599 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81994) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 14:57:35.827804 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81994) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 14:57:35.828454 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81994) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 14:57:35.828454 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81994) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 14:57:35.828712 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81994) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 14:57:35.842000 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81993) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 14:57:35.842517 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81993) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 14:57:35.844395 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81994) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 14:57:35.844681 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81994) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 14:57:35.846146 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81993) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 23 14:57:35.848575 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81994) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 23 14:57:35.849350 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81993) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 14:57:35.849646 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81993) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 14:57:35.851795 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81994) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 14:57:35.852131 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81994) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 14:57:35.854276 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81993) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 23 14:57:35.857177 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81994) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 23 14:57:35.857363 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81993) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 23 14:57:35.860320 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81994) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 23 14:57:35.860547 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81993) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 14:57:35.860848 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81993) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 14:57:35.861133 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81993) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 14:57:35.861414 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81993) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 14:57:35.861694 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81993) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 14:57:35.861974 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81993) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 14:57:35.863589 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81994) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 14:57:35.863889 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81994) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 14:57:35.864199 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81994) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 14:57:35.864490 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81994) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 14:57:35.864766 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81994) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 14:57:35.865047 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81994) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 14:57:35.878173 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81993) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 23 14:57:35.878470 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81993) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 14:57:35.878744 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81993) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 14:57:35.879011 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81993) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 14:57:35.881466 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81994) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 23 14:57:35.881759 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81994) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 14:57:35.882061 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81994) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 14:57:35.882325 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81994) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 14:57:35.885871 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81993) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 14:57:35.887383 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81993) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 14:57:35.888901 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81993) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 14:57:35.889181 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81993) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 14:57:35.889336 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81994) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 14:57:35.890852 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81994) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 14:57:35.891910 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81993) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 23 14:57:35.892233 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81993) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 14:57:35.892421 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81994) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 14:57:35.892517 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81993) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 14:57:35.892696 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81994) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 14:57:35.892799 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81993) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 14:57:35.893166 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81993) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 14:57:35.895389 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81994) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 23 14:57:35.895756 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81994) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 14:57:35.896079 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81994) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 14:57:35.896369 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81994) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 14:57:35.896829 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81994) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 14:57:35.904445 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81993) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 23 14:57:35.904732 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81993) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 14:57:35.905019 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81993) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 14:57:35.905301 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81993) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 14:57:35.905684 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81993) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 14:57:35.905966 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81993) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 14:57:35.906247 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81993) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 14:57:35.908366 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81994) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 23 14:57:35.908656 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81994) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 14:57:35.912861 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81994) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 14:57:35.912861 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81994) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 14:57:35.912861 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81994) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 14:57:35.912861 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81994) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 14:57:35.912861 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81994) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 14:57:35.922485 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81993) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 23 14:57:35.925433 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81993) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 23 14:57:35.926395 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81994) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 23 14:57:35.928313 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81993) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 14:57:35.929351 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81994) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 23 14:57:35.929770 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81993) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 14:57:35.931868 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81993) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 23 14:57:35.932114 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81993) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 14:57:35.932531 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81993) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 14:57:35.933170 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81994) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 14:57:35.934624 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81994) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 14:57:35.936168 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81994) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 23 14:57:35.936456 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81994) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 14:57:35.936855 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81994) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 14:57:35.938873 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81993) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 14:57:35.940353 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81993) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 23 14:57:35.948387 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81993) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 14:57:35.948491 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81994) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 14:57:35.948564 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81993) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 14:57:35.948564 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81993) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 14:57:35.948564 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81993) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 14:57:35.948782 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81994) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 23 14:57:35.948782 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81994) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 14:57:35.948782 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81994) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 14:57:35.948782 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81994) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 14:57:35.948782 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81994) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 14:57:35.952215 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81993) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 23 14:57:35.952509 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81993) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 14:57:35.952792 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81993) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 14:57:35.956395 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81994) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 23 14:57:35.956687 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81994) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 14:57:35.956970 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81994) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 14:57:35.959660 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81993) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 23 14:57:35.959954 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81993) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 14:57:35.960262 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81993) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 14:57:35.960544 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81993) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 14:57:35.960816 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81993) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 14:57:35.961086 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81993) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 14:57:35.961367 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81993) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 14:57:35.961634 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81993) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 14:57:35.961920 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81993) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 14:57:35.962215 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81993) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 14:57:35.962534 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81993) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 14:57:35.963018 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81993) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 14:57:35.963336 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81993) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 14:57:35.963598 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81993) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 14:57:35.964007 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81994) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 23 14:57:35.964303 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81994) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 14:57:35.964591 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81994) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 14:57:35.964876 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81994) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 14:57:35.965163 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81994) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 14:57:35.965485 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81994) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 14:57:35.965769 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81994) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 14:57:35.966051 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81994) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 14:57:35.966344 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81994) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 14:57:35.966617 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81994) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 14:57:35.966898 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81994) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 14:57:35.967162 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81994) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 14:57:35.967429 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81994) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 14:57:35.967698 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81994) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 14:57:35.998946 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x556a386dc9c0 pid: 81993 (default app) Sep 23 14:57:36.002434 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6193db3-9f19-4233-9390-7febf3d61d72 None None] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:57:36.002721 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6193db3-9f19-4233-9390-7febf3d61d72 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:57:36.002995 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6193db3-9f19-4233-9390-7febf3d61d72 None None] PATH_INFO: `/v3` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:57:36.003280 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x556a386dc9c0 pid: 81994 (default app) Sep 23 14:57:36.007186 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1/1] 10.222.0.55 () {50 vars in 791 bytes} [Fri Sep 23 14:57:35 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:57:43.982649 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aca8f5f-c0b3-4988-93bf-3e4cf90d57be None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:57:43.983195 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aca8f5f-c0b3-4988-93bf-3e4cf90d57be None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:57:43.983195 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aca8f5f-c0b3-4988-93bf-3e4cf90d57be None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:57:43.987728 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1/2] 10.222.0.55 () {54 vars in 927 bytes} [Fri Sep 23 14:57:43 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 23 14:57:43.995258 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b146c55f-78ce-4a04-b6d9-1862771ab71d None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:57:43.995795 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b146c55f-78ce-4a04-b6d9-1862771ab71d None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:57:43.995892 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b146c55f-78ce-4a04-b6d9-1862771ab71d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:57:44.078782 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: INFO dbcounter [None req-b146c55f-78ce-4a04-b6d9-1862771ab71d None None] Registered counter for database keystone Sep 23 14:57:44.101082 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG oslo_db.sqlalchemy.engines [None req-b146c55f-78ce-4a04-b6d9-1862771ab71d None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81993) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 23 14:57:44.108347 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writer thread running {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 23 14:57:44.297079 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG passlib.handlers.bcrypt [None req-b146c55f-78ce-4a04-b6d9-1862771ab71d None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81993) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 23 14:57:44.297388 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG passlib.handlers.bcrypt [None req-b146c55f-78ce-4a04-b6d9-1862771ab71d None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81993) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 23 14:57:44.357669 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-b146c55f-78ce-4a04-b6d9-1862771ab71d None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 14:57:44.359374 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-b146c55f-78ce-4a04-b6d9-1862771ab71d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 14:57:44.490089 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 2/3] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Sep 23 14:57:43 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 497 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Sep 23 14:57:44.988801 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aca8f5f-c0b3-4988-93bf-3e4cf90d57be None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:57:45.060136 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: INFO dbcounter [None req-6aca8f5f-c0b3-4988-93bf-3e4cf90d57be None None] Registered counter for database keystone Sep 23 14:57:45.081985 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG oslo_db.sqlalchemy.engines [None req-6aca8f5f-c0b3-4988-93bf-3e4cf90d57be None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81994) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 23 14:57:45.089097 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writer thread running {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 23 14:57:45.211111 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aca8f5f-c0b3-4988-93bf-3e4cf90d57be None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:57:45.216275 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6505bb2-ebfa-4331-b0c2-69c5cbcb5610 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:57:45.216908 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6505bb2-ebfa-4331-b0c2-69c5cbcb5610 admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:57:45.217228 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6505bb2-ebfa-4331-b0c2-69c5cbcb5610 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:57:45.217607 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6505bb2-ebfa-4331-b0c2-69c5cbcb5610 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:57:45.228055 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6505bb2-ebfa-4331-b0c2-69c5cbcb5610 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:57:45.292886 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: WARNING keystone.common.rbac_enforcer.enforcer [None req-c6505bb2-ebfa-4331-b0c2-69c5cbcb5610 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 23 14:57:45.292886 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6505bb2-ebfa-4331-b0c2-69c5cbcb5610 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:57:45.295040 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 2/4] 10.222.0.55 () {58 vars in 1324 bytes} [Fri Sep 23 14:57:44 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 23 14:57:45.301314 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b146c55f-78ce-4a04-b6d9-1862771ab71d None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:57:45.319021 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b146c55f-78ce-4a04-b6d9-1862771ab71d None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:57:45.324132 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b637e6ed-f98f-4a2c-be81-7d4342f6d65d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:57:45.325251 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b637e6ed-f98f-4a2c-be81-7d4342f6d65d admin admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:57:45.325538 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b637e6ed-f98f-4a2c-be81-7d4342f6d65d admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:57:45.325814 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b637e6ed-f98f-4a2c-be81-7d4342f6d65d admin admin] PATH_INFO: `/v3/projects/23b5c58d198d4456bfeec0a8daae1113` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:57:45.327405 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b637e6ed-f98f-4a2c-be81-7d4342f6d65d admin admin] RBAC: Authorizing `identity:get_project(project_id=23b5c58d198d4456bfeec0a8daae1113)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:57:45.379672 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: WARNING keystone.common.rbac_enforcer.enforcer [None req-b637e6ed-f98f-4a2c-be81-7d4342f6d65d admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 23 14:57:45.380036 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b637e6ed-f98f-4a2c-be81-7d4342f6d65d admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:57:45.385314 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 3/5] 10.222.0.55 () {56 vars in 1207 bytes} [Fri Sep 23 14:57:45 2022] GET /identity/v3/projects/23b5c58d198d4456bfeec0a8daae1113 => generated 344 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:57:47.641664 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec4e6a2-1ead-48a3-a2c4-aab5f3fbd8c9 None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:57:47.642336 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec4e6a2-1ead-48a3-a2c4-aab5f3fbd8c9 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:57:47.642336 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec4e6a2-1ead-48a3-a2c4-aab5f3fbd8c9 None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:57:47.646557 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 3/6] 10.222.0.55 () {54 vars in 927 bytes} [Fri Sep 23 14:57:47 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 14:57:47.652893 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbac56d-33ca-47d6-b58d-ff2e49dd7469 None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:57:47.653288 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbac56d-33ca-47d6-b58d-ff2e49dd7469 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:57:47.653464 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbac56d-33ca-47d6-b58d-ff2e49dd7469 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:57:47.674651 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-9bbac56d-33ca-47d6-b58d-ff2e49dd7469 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 14:57:47.676224 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-9bbac56d-33ca-47d6-b58d-ff2e49dd7469 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 14:57:47.684167 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 4/7] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Sep 23 14:57:47 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Sep 23 14:57:47.956679 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec4e6a2-1ead-48a3-a2c4-aab5f3fbd8c9 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:57:47.970247 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec4e6a2-1ead-48a3-a2c4-aab5f3fbd8c9 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:57:47.974467 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61cdfc9-1f26-48c0-9a03-4c8b16da6970 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:57:47.975076 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61cdfc9-1f26-48c0-9a03-4c8b16da6970 admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:57:47.975361 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61cdfc9-1f26-48c0-9a03-4c8b16da6970 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:57:47.975637 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61cdfc9-1f26-48c0-9a03-4c8b16da6970 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:57:47.983722 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61cdfc9-1f26-48c0-9a03-4c8b16da6970 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:57:47.985537 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61cdfc9-1f26-48c0-9a03-4c8b16da6970 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:57:47.994401 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 4/8] 10.222.0.55 () {58 vars in 1324 bytes} [Fri Sep 23 14:57:47 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 23 14:57:48.000344 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbac56d-33ca-47d6-b58d-ff2e49dd7469 None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:57:48.011152 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbac56d-33ca-47d6-b58d-ff2e49dd7469 None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:57:48.018212 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ce6d0a-2da5-4c01-a3a6-9e0f03999d7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:57:48.018844 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ce6d0a-2da5-4c01-a3a6-9e0f03999d7f admin admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:57:48.019128 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ce6d0a-2da5-4c01-a3a6-9e0f03999d7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:57:48.023256 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ce6d0a-2da5-4c01-a3a6-9e0f03999d7f admin admin] PATH_INFO: `/v3/users/0fc10e0211a54a0faf7ea9b86542de03` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:57:48.025827 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ce6d0a-2da5-4c01-a3a6-9e0f03999d7f admin admin] RBAC: Authorizing `identity:get_user(user_id=0fc10e0211a54a0faf7ea9b86542de03)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:57:48.027674 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ce6d0a-2da5-4c01-a3a6-9e0f03999d7f admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:57:48.031443 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 5/9] 10.222.0.55 () {56 vars in 1198 bytes} [Fri Sep 23 14:57:47 2022] GET /identity/v3/users/0fc10e0211a54a0faf7ea9b86542de03 => generated 247 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:57:50.284197 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c0785b-ff34-4ce2-87d2-18e3b01151e2 None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:57:50.284741 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c0785b-ff34-4ce2-87d2-18e3b01151e2 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:57:50.284741 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c0785b-ff34-4ce2-87d2-18e3b01151e2 None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:57:50.286082 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 5/10] 10.222.0.55 () {54 vars in 927 bytes} [Fri Sep 23 14:57:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 14:57:50.292117 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3d6ac8-0114-4d8e-a110-71d0e32ca767 None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:57:50.292508 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3d6ac8-0114-4d8e-a110-71d0e32ca767 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:57:50.292791 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3d6ac8-0114-4d8e-a110-71d0e32ca767 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:57:50.312577 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-ee3d6ac8-0114-4d8e-a110-71d0e32ca767 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 14:57:50.313883 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-ee3d6ac8-0114-4d8e-a110-71d0e32ca767 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 14:57:50.344946 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a31395-d9d0-4ac3-a89a-eeb3de829e79 None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:57:50.345233 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a31395-d9d0-4ac3-a89a-eeb3de829e79 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:57:50.345613 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a31395-d9d0-4ac3-a89a-eeb3de829e79 None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:57:50.346933 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 6/11] 10.222.0.55 () {54 vars in 927 bytes} [Fri Sep 23 14:57:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 23 14:57:50.352019 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d44800-c668-459e-a5dc-993c9cb12fe1 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:57:50.352311 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d44800-c668-459e-a5dc-993c9cb12fe1 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:57:50.352599 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d44800-c668-459e-a5dc-993c9cb12fe1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:57:50.400941 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 6/12] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Sep 23 14:57:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 23 14:57:50.401870 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG passlib.handlers.bcrypt [None req-27d44800-c668-459e-a5dc-993c9cb12fe1 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81994) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 23 14:57:50.402424 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG passlib.handlers.bcrypt [None req-27d44800-c668-459e-a5dc-993c9cb12fe1 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81994) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 23 14:57:50.456755 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-27d44800-c668-459e-a5dc-993c9cb12fe1 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 14:57:50.458588 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-27d44800-c668-459e-a5dc-993c9cb12fe1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 14:57:50.563486 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 7/13] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Sep 23 14:57:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 23 14:57:50.673370 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3d6ac8-0114-4d8e-a110-71d0e32ca767 None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:57:50.697865 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:57:50.777674 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3d6ac8-0114-4d8e-a110-71d0e32ca767 None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:57:50.811916 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1844e9c-c184-447d-86e6-fc27aae3a6a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:57:50.812576 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1844e9c-c184-447d-86e6-fc27aae3a6a5 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:57:50.812856 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1844e9c-c184-447d-86e6-fc27aae3a6a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:57:50.813130 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1844e9c-c184-447d-86e6-fc27aae3a6a5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:57:50.823904 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1844e9c-c184-447d-86e6-fc27aae3a6a5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 14:57:50.832613 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application [None req-f1844e9c-c184-447d-86e6-fc27aae3a6a5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 23 14:57:50.832613 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 14:57:50.832613 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 14:57:50.832613 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 14:57:50.832613 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 14:57:50.832613 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 14:57:50.832613 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 14:57:50.832613 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 14:57:50.832613 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 14:57:50.832613 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 14:57:50.832613 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 14:57:50.832613 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 14:57:50.832613 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 14:57:50.832613 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 14:57:50.832613 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 14:57:50.832613 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 14:57:50.833593 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 14:57:50.833593 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 14:57:50.833593 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 14:57:50.833593 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 14:57:50.833593 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 14:57:50.833593 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 14:57:50.833593 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 14:57:50.833593 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 14:57:50.833593 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 14:57:50.833593 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 14:57:50.833593 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 14:57:50.833593 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application with Lock( Sep 23 14:57:50.833593 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 14:57:50.833593 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._enter() Sep 23 14:57:50.833593 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 14:57:50.833593 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 14:57:50.834682 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 14:57:50.834682 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.creator() Sep 23 14:57:50.834682 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 14:57:50.834682 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application created_value = creator( Sep 23 14:57:50.834682 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 14:57:50.834682 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 14:57:50.834682 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 14:57:50.834682 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 14:57:50.834682 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 14:57:50.834682 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 14:57:50.834682 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 23 14:57:50.834682 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Sep 23 14:57:50.834682 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 7/14] 10.222.0.55 () {56 vars in 1096 bytes} [Fri Sep 23 14:57:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 14:57:50.839620 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d44800-c668-459e-a5dc-993c9cb12fe1 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:57:50.856935 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1844e9c-c184-447d-86e6-fc27aae3a6a5 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:57:50.941549 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:57:50.946796 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d44800-c668-459e-a5dc-993c9cb12fe1 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:57:50.949461 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:57:50.960357 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1844e9c-c184-447d-86e6-fc27aae3a6a5 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:57:50.985507 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e73829-8ce4-4d80-8c77-1f1d3969bac2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:57:50.987077 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e73829-8ce4-4d80-8c77-1f1d3969bac2 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:57:50.987370 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e73829-8ce4-4d80-8c77-1f1d3969bac2 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:57:50.987647 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e73829-8ce4-4d80-8c77-1f1d3969bac2 None admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:57:50.988527 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e73829-8ce4-4d80-8c77-1f1d3969bac2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:57:50.988832 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e73829-8ce4-4d80-8c77-1f1d3969bac2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:57:50.990275 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e73829-8ce4-4d80-8c77-1f1d3969bac2 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:57:50.998095 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad45bd6b-0512-4869-848f-0c15cdf0b9fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:57:50.998796 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad45bd6b-0512-4869-848f-0c15cdf0b9fb None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:57:50.999169 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad45bd6b-0512-4869-848f-0c15cdf0b9fb None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:57:50.999516 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad45bd6b-0512-4869-848f-0c15cdf0b9fb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:57:51.003441 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 8/15] 10.222.0.55 () {56 vars in 1099 bytes} [Fri Sep 23 14:57:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:57:51.009697 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e73829-8ce4-4d80-8c77-1f1d3969bac2 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:57:51.072744 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad45bd6b-0512-4869-848f-0c15cdf0b9fb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:57:51.074242 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad45bd6b-0512-4869-848f-0c15cdf0b9fb None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:57:51.121359 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e73829-8ce4-4d80-8c77-1f1d3969bac2 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:57:51.156260 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a1ca2f-faef-46ad-98fa-cef906ac0090 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:57:51.157374 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a1ca2f-faef-46ad-98fa-cef906ac0090 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:57:51.157657 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a1ca2f-faef-46ad-98fa-cef906ac0090 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:57:51.157924 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a1ca2f-faef-46ad-98fa-cef906ac0090 None admin] PATH_INFO: `/v3/users/0fc10e0211a54a0faf7ea9b86542de03` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:57:51.159398 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a1ca2f-faef-46ad-98fa-cef906ac0090 None admin] RBAC: Authorizing `identity:get_user(user_id=0fc10e0211a54a0faf7ea9b86542de03)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:57:51.160113 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:57:51.160934 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a1ca2f-faef-46ad-98fa-cef906ac0090 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:57:51.162488 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 9/16] 10.222.0.55 () {56 vars in 1177 bytes} [Fri Sep 23 14:57:51 2022] GET /identity/v3/users/0fc10e0211a54a0faf7ea9b86542de03 => generated 247 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 14:57:51.168662 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a1ca2f-faef-46ad-98fa-cef906ac0090 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:57:51.179424 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 8/17] 10.222.0.55 () {58 vars in 1282 bytes} [Fri Sep 23 14:57:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 23 14:57:51.184900 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad45bd6b-0512-4869-848f-0c15cdf0b9fb None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:57:51.216907 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:57:51.270344 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a1ca2f-faef-46ad-98fa-cef906ac0090 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:57:51.287151 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad45bd6b-0512-4869-848f-0c15cdf0b9fb None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:57:51.306759 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55238345-f64b-4805-837b-3e57b1b4c46a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:57:51.307419 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55238345-f64b-4805-837b-3e57b1b4c46a None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:57:51.307709 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55238345-f64b-4805-837b-3e57b1b4c46a None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:57:51.308016 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55238345-f64b-4805-837b-3e57b1b4c46a None admin] PATH_INFO: `/v3/domains/default` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:57:51.309170 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55238345-f64b-4805-837b-3e57b1b4c46a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:57:51.310724 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55238345-f64b-4805-837b-3e57b1b4c46a None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:57:51.312419 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 10/18] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Sep 23 14:57:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 14:57:51.318094 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55238345-f64b-4805-837b-3e57b1b4c46a None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:57:51.323289 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445825db-8bd8-47e1-868f-2928b6598e0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:57:51.323937 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445825db-8bd8-47e1-868f-2928b6598e0f None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:57:51.324249 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445825db-8bd8-47e1-868f-2928b6598e0f None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:57:51.324526 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445825db-8bd8-47e1-868f-2928b6598e0f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:57:51.335092 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445825db-8bd8-47e1-868f-2928b6598e0f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:57:51.336806 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445825db-8bd8-47e1-868f-2928b6598e0f None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:57:51.355913 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application [None req-445825db-8bd8-47e1-868f-2928b6598e0f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 14:57:51.355913 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 14:57:51.355913 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 14:57:51.355913 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 14:57:51.355913 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 14:57:51.355913 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 14:57:51.355913 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 14:57:51.355913 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 14:57:51.355913 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 14:57:51.355913 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 14:57:51.355913 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 14:57:51.355913 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 14:57:51.355913 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 14:57:51.355913 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 14:57:51.355913 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 14:57:51.355913 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 14:57:51.357021 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 14:57:51.357021 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 14:57:51.357021 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 14:57:51.357021 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 14:57:51.357021 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 14:57:51.357021 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 14:57:51.357021 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 14:57:51.357021 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application with Lock( Sep 23 14:57:51.357021 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 14:57:51.357021 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._enter() Sep 23 14:57:51.357021 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 14:57:51.357021 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 14:57:51.357021 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 14:57:51.357021 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.creator() Sep 23 14:57:51.357021 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 14:57:51.357021 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application created_value = creator( Sep 23 14:57:51.357021 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 14:57:51.357894 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 14:57:51.357894 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 14:57:51.357894 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Sep 23 14:57:51.357894 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 9/19] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Sep 23 14:57:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 23 14:57:51.362942 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445825db-8bd8-47e1-868f-2928b6598e0f None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:57:51.398952 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:57:51.429296 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55238345-f64b-4805-837b-3e57b1b4c46a None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:57:51.431650 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:57:51.466765 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8e3347-a390-4000-b195-704f1ae5cc65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:57:51.467521 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8e3347-a390-4000-b195-704f1ae5cc65 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:57:51.467826 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8e3347-a390-4000-b195-704f1ae5cc65 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:57:51.467931 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445825db-8bd8-47e1-868f-2928b6598e0f None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:57:51.468195 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8e3347-a390-4000-b195-704f1ae5cc65 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:57:51.468923 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8e3347-a390-4000-b195-704f1ae5cc65 None admin] RBAC: Adding query filter params (role.id=8e2919e79df040e8926f21607818b58c, scope.domain.id=default, user.id=0fc10e0211a54a0faf7ea9b86542de03) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:57:51.469239 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8e3347-a390-4000-b195-704f1ae5cc65 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:57:51.470679 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8e3347-a390-4000-b195-704f1ae5cc65 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:57:51.479138 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 11/20] 10.222.0.55 () {56 vars in 1322 bytes} [Fri Sep 23 14:57:51 2022] GET /identity/v3/role_assignments?user.id=0fc10e0211a54a0faf7ea9b86542de03&scope.domain.id=default&role.id=8e2919e79df040e8926f21607818b58c => generated 233 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 14:57:51.504032 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd709749-f31a-4e75-bbd6-a5532f0cd320 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:57:51.504930 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd709749-f31a-4e75-bbd6-a5532f0cd320 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:57:51.505215 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd709749-f31a-4e75-bbd6-a5532f0cd320 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:57:51.505492 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd709749-f31a-4e75-bbd6-a5532f0cd320 None admin] PATH_INFO: `/v3/domains` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:57:51.506165 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd709749-f31a-4e75-bbd6-a5532f0cd320 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:57:51.506474 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd709749-f31a-4e75-bbd6-a5532f0cd320 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:57:51.507911 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd709749-f31a-4e75-bbd6-a5532f0cd320 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:57:51.527929 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 10/21] 10.222.0.55 () {56 vars in 1109 bytes} [Fri Sep 23 14:57:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 14:57:53.950217 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eda9512-2fa4-43cf-8221-d37a318d5e40 None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:57:53.951127 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eda9512-2fa4-43cf-8221-d37a318d5e40 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:57:53.951588 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eda9512-2fa4-43cf-8221-d37a318d5e40 None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:57:53.953180 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 12/22] 10.222.0.55 () {54 vars in 927 bytes} [Fri Sep 23 14:57:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 14:57:53.958862 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f5a7a1-e2c2-4388-902e-32fdafcec9bd None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:57:53.959744 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f5a7a1-e2c2-4388-902e-32fdafcec9bd None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:57:53.960235 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f5a7a1-e2c2-4388-902e-32fdafcec9bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:57:53.998898 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-39f5a7a1-e2c2-4388-902e-32fdafcec9bd None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 14:57:54.000412 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-39f5a7a1-e2c2-4388-902e-32fdafcec9bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 14:57:54.128548 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 11/23] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Sep 23 14:57:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 23 14:57:54.203615 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2980905-e4e0-4b20-b343-0e830a7a9cd4 None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:57:54.204246 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2980905-e4e0-4b20-b343-0e830a7a9cd4 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:57:54.204744 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2980905-e4e0-4b20-b343-0e830a7a9cd4 None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:57:54.206593 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 13/24] 10.222.0.55 () {54 vars in 927 bytes} [Fri Sep 23 14:57:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 14:57:54.214365 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff35abb6-b0d8-4528-b678-6c944a1c5753 None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:57:54.214968 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff35abb6-b0d8-4528-b678-6c944a1c5753 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:57:54.215445 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff35abb6-b0d8-4528-b678-6c944a1c5753 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:57:54.245865 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-ff35abb6-b0d8-4528-b678-6c944a1c5753 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 14:57:54.247433 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-ff35abb6-b0d8-4528-b678-6c944a1c5753 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 14:57:54.271216 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:57:54.333934 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 12/25] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Sep 23 14:57:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 14:57:54.404439 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47185ba5-8b84-41ff-8788-fe333095a794 None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:57:54.405111 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47185ba5-8b84-41ff-8788-fe333095a794 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:57:54.405592 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47185ba5-8b84-41ff-8788-fe333095a794 None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:57:54.407642 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 14/26] 10.222.0.55 () {54 vars in 927 bytes} [Fri Sep 23 14:57:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 14:57:54.413406 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ec5ae0-9183-4d96-be4f-34ff438d206f None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:57:54.413927 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ec5ae0-9183-4d96-be4f-34ff438d206f None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:57:54.414404 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ec5ae0-9183-4d96-be4f-34ff438d206f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:57:54.439216 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-09ec5ae0-9183-4d96-be4f-34ff438d206f None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 14:57:54.440837 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-09ec5ae0-9183-4d96-be4f-34ff438d206f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 14:57:54.539282 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 13/27] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Sep 23 14:57:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 14:57:54.578658 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47185ba5-8b84-41ff-8788-fe333095a794 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:57:54.685999 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c75ec29-59d5-463e-bc72-ae3c6b1f7b89 None None] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:57:54.686720 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c75ec29-59d5-463e-bc72-ae3c6b1f7b89 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:57:54.687200 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c75ec29-59d5-463e-bc72-ae3c6b1f7b89 None None] PATH_INFO: `/` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:57:54.688933 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 14/28] 10.222.0.55 () {54 vars in 927 bytes} [Fri Sep 23 14:57:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 23 14:57:54.701834 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cab7494-647c-4830-b9d4-8e34b9aeb127 None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:57:54.702517 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cab7494-647c-4830-b9d4-8e34b9aeb127 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:57:54.703008 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cab7494-647c-4830-b9d4-8e34b9aeb127 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:57:54.716023 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47185ba5-8b84-41ff-8788-fe333095a794 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:57:54.737060 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-4cab7494-647c-4830-b9d4-8e34b9aeb127 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 14:57:54.738715 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-4cab7494-647c-4830-b9d4-8e34b9aeb127 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 14:57:54.755432 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1091031a-9a6c-4bbf-a40b-b2e36e8129dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:57:54.756132 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1091031a-9a6c-4bbf-a40b-b2e36e8129dd None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:57:54.756421 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1091031a-9a6c-4bbf-a40b-b2e36e8129dd None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:57:54.756702 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1091031a-9a6c-4bbf-a40b-b2e36e8129dd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:57:54.776836 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1091031a-9a6c-4bbf-a40b-b2e36e8129dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 14:57:54.788932 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-1091031a-9a6c-4bbf-a40b-b2e36e8129dd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 23 14:57:54.788932 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 14:57:54.788932 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 14:57:54.788932 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 14:57:54.788932 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 14:57:54.788932 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 14:57:54.788932 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 14:57:54.788932 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 14:57:54.788932 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 14:57:54.788932 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 14:57:54.788932 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 14:57:54.788932 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 14:57:54.788932 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 14:57:54.788932 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 14:57:54.788932 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 14:57:54.788932 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 14:57:54.789857 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 14:57:54.789857 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 14:57:54.789857 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 14:57:54.789857 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 14:57:54.789857 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 14:57:54.789857 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 14:57:54.789857 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 14:57:54.789857 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 14:57:54.789857 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 14:57:54.789857 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 14:57:54.789857 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 14:57:54.789857 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Sep 23 14:57:54.789857 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 14:57:54.789857 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Sep 23 14:57:54.789857 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 14:57:54.789857 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 14:57:54.791910 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 14:57:54.791910 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Sep 23 14:57:54.791910 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 14:57:54.791910 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Sep 23 14:57:54.791910 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 14:57:54.791910 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 14:57:54.791910 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 14:57:54.791910 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 14:57:54.791910 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 14:57:54.791910 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 14:57:54.791910 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 23 14:57:54.791910 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Sep 23 14:57:54.791910 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 15/29] 10.222.0.55 () {56 vars in 1096 bytes} [Fri Sep 23 14:57:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 14:57:54.799017 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1091031a-9a6c-4bbf-a40b-b2e36e8129dd None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:57:54.852444 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:57:54.874100 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 15/30] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Sep 23 14:57:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 23 14:57:54.910370 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cab7494-647c-4830-b9d4-8e34b9aeb127 None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:57:54.937130 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:57:54.986078 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1091031a-9a6c-4bbf-a40b-b2e36e8129dd None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:57:55.022850 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ad07cf-0af1-4395-b41a-55fe3f5b6c0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:57:55.023478 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ad07cf-0af1-4395-b41a-55fe3f5b6c0c None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:57:55.023763 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ad07cf-0af1-4395-b41a-55fe3f5b6c0c None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:57:55.024072 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ad07cf-0af1-4395-b41a-55fe3f5b6c0c None admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:57:55.025408 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ad07cf-0af1-4395-b41a-55fe3f5b6c0c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:57:55.027493 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ad07cf-0af1-4395-b41a-55fe3f5b6c0c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:57:55.029016 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ad07cf-0af1-4395-b41a-55fe3f5b6c0c None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:57:55.039158 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 16/31] 10.222.0.55 () {56 vars in 1099 bytes} [Fri Sep 23 14:57:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 14:57:55.042499 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ad07cf-0af1-4395-b41a-55fe3f5b6c0c None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:57:55.135442 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cab7494-647c-4830-b9d4-8e34b9aeb127 None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:57:55.213314 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:57:55.214860 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a34b60b-6a4b-40a6-9dfd-aec42507c862 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:57:55.215545 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a34b60b-6a4b-40a6-9dfd-aec42507c862 None admin] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:57:55.215831 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a34b60b-6a4b-40a6-9dfd-aec42507c862 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:57:55.216140 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a34b60b-6a4b-40a6-9dfd-aec42507c862 None admin] PATH_INFO: `/v3/roles` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:57:55.216893 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a34b60b-6a4b-40a6-9dfd-aec42507c862 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:57:55.218400 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a34b60b-6a4b-40a6-9dfd-aec42507c862 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:57:55.231440 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ad07cf-0af1-4395-b41a-55fe3f5b6c0c None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:57:55.267700 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a2eb1a-b365-46b5-9cd5-c574bcc7d252 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:57:55.268476 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a2eb1a-b365-46b5-9cd5-c574bcc7d252 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:57:55.268760 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a2eb1a-b365-46b5-9cd5-c574bcc7d252 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:57:55.269047 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a2eb1a-b365-46b5-9cd5-c574bcc7d252 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:57:55.287933 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a2eb1a-b365-46b5-9cd5-c574bcc7d252 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:57:55.289428 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a2eb1a-b365-46b5-9cd5-c574bcc7d252 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:57:55.305789 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-31a2eb1a-b365-46b5-9cd5-c574bcc7d252 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 14:57:55.305789 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 14:57:55.305789 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 14:57:55.305789 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 14:57:55.305789 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 14:57:55.305789 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 14:57:55.305789 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 14:57:55.305789 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 14:57:55.305789 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 14:57:55.305789 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 14:57:55.305789 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 14:57:55.305789 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 14:57:55.305789 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 14:57:55.305789 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 14:57:55.305789 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 14:57:55.305789 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 14:57:55.306755 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 14:57:55.306755 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 14:57:55.306755 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 14:57:55.306755 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 14:57:55.306755 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 14:57:55.306755 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 14:57:55.306755 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 14:57:55.306755 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Sep 23 14:57:55.306755 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 14:57:55.306755 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Sep 23 14:57:55.306755 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 14:57:55.306755 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 14:57:55.306755 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 14:57:55.306755 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Sep 23 14:57:55.306755 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 14:57:55.306755 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Sep 23 14:57:55.306755 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 14:57:55.308246 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 14:57:55.308246 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 14:57:55.308246 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Sep 23 14:57:55.308246 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 17/32] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Sep 23 14:57:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 14:57:55.308246 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a2eb1a-b365-46b5-9cd5-c574bcc7d252 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:57:55.423876 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a2eb1a-b365-46b5-9cd5-c574bcc7d252 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:57:55.465353 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c87b57-6e8f-4f60-8d30-a0be187e63a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:57:55.466091 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c87b57-6e8f-4f60-8d30-a0be187e63a3 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:57:55.466399 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c87b57-6e8f-4f60-8d30-a0be187e63a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:57:55.466712 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c87b57-6e8f-4f60-8d30-a0be187e63a3 None admin] PATH_INFO: `/v3/users/0fc10e0211a54a0faf7ea9b86542de03` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:57:55.467914 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c87b57-6e8f-4f60-8d30-a0be187e63a3 None admin] RBAC: Authorizing `identity:get_user(user_id=0fc10e0211a54a0faf7ea9b86542de03)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:57:55.469479 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c87b57-6e8f-4f60-8d30-a0be187e63a3 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:57:55.471258 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 16/33] 10.222.0.55 () {60 vars in 1131 bytes} [Fri Sep 23 14:57:54 2022] POST /identity/v3/roles => generated 219 bytes in 561 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 14:57:55.471461 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 18/34] 10.222.0.55 () {56 vars in 1177 bytes} [Fri Sep 23 14:57:55 2022] GET /identity/v3/users/0fc10e0211a54a0faf7ea9b86542de03 => generated 247 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:57:55.472227 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c87b57-6e8f-4f60-8d30-a0be187e63a3 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:57:55.478159 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a34b60b-6a4b-40a6-9dfd-aec42507c862 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:57:55.486675 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:57:55.633375 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:57:55.671469 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a34b60b-6a4b-40a6-9dfd-aec42507c862 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:57:55.683701 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c87b57-6e8f-4f60-8d30-a0be187e63a3 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:57:55.720093 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d508f11f-ce2e-4d8c-9e70-7561bb9163a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:57:55.721028 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d508f11f-ce2e-4d8c-9e70-7561bb9163a6 None admin] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:57:55.721481 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d508f11f-ce2e-4d8c-9e70-7561bb9163a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:57:55.721905 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d508f11f-ce2e-4d8c-9e70-7561bb9163a6 None admin] PATH_INFO: `/v3/roles` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:57:55.722889 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d508f11f-ce2e-4d8c-9e70-7561bb9163a6 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:57:55.724840 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d508f11f-ce2e-4d8c-9e70-7561bb9163a6 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:57:55.729623 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af081957-3695-458b-9c88-8dbb8007f721 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:57:55.730456 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af081957-3695-458b-9c88-8dbb8007f721 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:57:55.730829 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af081957-3695-458b-9c88-8dbb8007f721 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:57:55.731200 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af081957-3695-458b-9c88-8dbb8007f721 None admin] PATH_INFO: `/v3/domains` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:57:55.731961 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af081957-3695-458b-9c88-8dbb8007f721 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:57:55.732384 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af081957-3695-458b-9c88-8dbb8007f721 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:57:55.734085 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af081957-3695-458b-9c88-8dbb8007f721 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:57:55.753316 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 19/35] 10.222.0.55 () {56 vars in 1109 bytes} [Fri Sep 23 14:57:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:57:55.754175 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af081957-3695-458b-9c88-8dbb8007f721 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:57:55.823057 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 17/36] 10.222.0.55 () {60 vars in 1131 bytes} [Fri Sep 23 14:57:55 2022] POST /identity/v3/roles => generated 225 bytes in 351 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 14:57:55.823929 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d508f11f-ce2e-4d8c-9e70-7561bb9163a6 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:57:55.836720 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:57:55.879694 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af081957-3695-458b-9c88-8dbb8007f721 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:57:55.924295 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b0495b-8629-4247-bc80-f77466b304cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:57:55.925472 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b0495b-8629-4247-bc80-f77466b304cd None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:57:55.925978 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b0495b-8629-4247-bc80-f77466b304cd None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:57:55.926469 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b0495b-8629-4247-bc80-f77466b304cd None admin] PATH_INFO: `/v3/domains/default` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:57:55.928185 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b0495b-8629-4247-bc80-f77466b304cd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:57:55.930331 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b0495b-8629-4247-bc80-f77466b304cd None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:57:55.932800 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 20/37] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Sep 23 14:57:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:57:55.934279 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad169e24-5e3b-4310-8f21-e3a22a321200 None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:57:55.934775 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad169e24-5e3b-4310-8f21-e3a22a321200 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:57:55.935274 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad169e24-5e3b-4310-8f21-e3a22a321200 None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:57:55.937035 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 21/38] 10.222.0.55 () {54 vars in 927 bytes} [Fri Sep 23 14:57:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 14:57:55.942415 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad169e24-5e3b-4310-8f21-e3a22a321200 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:57:55.952964 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d508f11f-ce2e-4d8c-9e70-7561bb9163a6 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:57:55.996730 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52d311e-81b6-4574-bee2-548689bb4375 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:57:55.997383 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52d311e-81b6-4574-bee2-548689bb4375 None admin] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:57:55.997667 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52d311e-81b6-4574-bee2-548689bb4375 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:57:55.997945 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52d311e-81b6-4574-bee2-548689bb4375 None admin] PATH_INFO: `/v3/projects` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:57:55.998615 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52d311e-81b6-4574-bee2-548689bb4375 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:57:56.000144 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52d311e-81b6-4574-bee2-548689bb4375 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:57:56.006170 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:57:56.075200 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad169e24-5e3b-4310-8f21-e3a22a321200 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:57:56.112451 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ab90ae-7849-4be3-8bf6-4b0ef597c2b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:57:56.113262 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ab90ae-7849-4be3-8bf6-4b0ef597c2b2 None admin] REQUEST_METHOD: `PUT` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:57:56.113549 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ab90ae-7849-4be3-8bf6-4b0ef597c2b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:57:56.113831 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ab90ae-7849-4be3-8bf6-4b0ef597c2b2 None admin] PATH_INFO: `/v3/domains/default/users/0fc10e0211a54a0faf7ea9b86542de03/roles/8e2919e79df040e8926f21607818b58c` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:57:56.115780 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ab90ae-7849-4be3-8bf6-4b0ef597c2b2 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=0fc10e0211a54a0faf7ea9b86542de03, role_id=8e2919e79df040e8926f21607818b58c)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:57:56.117269 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ab90ae-7849-4be3-8bf6-4b0ef597c2b2 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:57:56.121368 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 18/39] 10.222.0.55 () {60 vars in 1141 bytes} [Fri Sep 23 14:57:55 2022] POST /identity/v3/projects => generated 301 bytes in 298 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 14:57:56.122881 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b54562-a784-4161-97bf-662fd9e3efd4 None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:57:56.128333 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b54562-a784-4161-97bf-662fd9e3efd4 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:57:56.128494 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:57:56.128597 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b54562-a784-4161-97bf-662fd9e3efd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:57:56.144250 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-a6b54562-a784-4161-97bf-662fd9e3efd4 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 14:57:56.145937 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-a6b54562-a784-4161-97bf-662fd9e3efd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 14:57:56.233087 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 19/40] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Sep 23 14:57:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 14:57:56.234343 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63cfe70-624a-45ea-b9e1-6b193b31b21c None None] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:57:56.234694 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63cfe70-624a-45ea-b9e1-6b193b31b21c None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:57:56.235000 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63cfe70-624a-45ea-b9e1-6b193b31b21c None None] PATH_INFO: `/` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:57:56.236685 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 20/41] 10.222.0.55 () {54 vars in 927 bytes} [Fri Sep 23 14:57:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 14:57:56.242494 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e376df63-27ce-4aa9-b350-a525a2f91193 None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:57:56.242823 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e376df63-27ce-4aa9-b350-a525a2f91193 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:57:56.243038 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e376df63-27ce-4aa9-b350-a525a2f91193 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:57:56.367571 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 22/42] 10.222.0.55 () {58 vars in 1361 bytes} [Fri Sep 23 14:57:55 2022] PUT /identity/v3/domains/default/users/0fc10e0211a54a0faf7ea9b86542de03/roles/8e2919e79df040e8926f21607818b58c => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 14:57:56.375681 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-e376df63-27ce-4aa9-b350-a525a2f91193 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 14:57:56.377012 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-e376df63-27ce-4aa9-b350-a525a2f91193 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 14:57:56.455507 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 21/43] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Sep 23 14:57:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 23 14:57:56.535547 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ab90ae-7849-4be3-8bf6-4b0ef597c2b2 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:57:56.651284 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ab90ae-7849-4be3-8bf6-4b0ef597c2b2 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:57:56.686125 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7d74e4-0e29-42c2-85fb-478c8ea587b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:57:56.686747 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7d74e4-0e29-42c2-85fb-478c8ea587b7 None admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:57:56.687032 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7d74e4-0e29-42c2-85fb-478c8ea587b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:57:56.687310 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7d74e4-0e29-42c2-85fb-478c8ea587b7 None admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:57:56.687997 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7d74e4-0e29-42c2-85fb-478c8ea587b7 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:57:56.689611 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7d74e4-0e29-42c2-85fb-478c8ea587b7 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:57:56.739352 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e376df63-27ce-4aa9-b350-a525a2f91193 None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:57:56.792900 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:57:56.796144 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 23/44] 10.222.0.55 () {60 vars in 1131 bytes} [Fri Sep 23 14:57:56 2022] POST /identity/v3/roles => generated 223 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 14:57:56.860254 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e376df63-27ce-4aa9-b350-a525a2f91193 None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:57:56.898905 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27a625a-0b13-4073-b30e-76c5e5c12dea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:57:56.899791 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27a625a-0b13-4073-b30e-76c5e5c12dea None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:57:56.900246 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27a625a-0b13-4073-b30e-76c5e5c12dea None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:57:56.900595 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27a625a-0b13-4073-b30e-76c5e5c12dea None admin] PATH_INFO: `/v3/domains/default` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:57:56.901891 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27a625a-0b13-4073-b30e-76c5e5c12dea None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:57:56.903418 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27a625a-0b13-4073-b30e-76c5e5c12dea None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:57:56.905184 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 22/45] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Sep 23 14:57:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 14:57:56.911229 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7d74e4-0e29-42c2-85fb-478c8ea587b7 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:57:56.976382 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:57:57.064535 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7d74e4-0e29-42c2-85fb-478c8ea587b7 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:57:57.070090 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4a8c12-9736-4ea6-96f1-cedec314c9b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:57:57.070725 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4a8c12-9736-4ea6-96f1-cedec314c9b8 None admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:57:57.071013 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4a8c12-9736-4ea6-96f1-cedec314c9b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:57:57.071296 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4a8c12-9736-4ea6-96f1-cedec314c9b8 None admin] PATH_INFO: `/v3/projects` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:57:57.072001 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4a8c12-9736-4ea6-96f1-cedec314c9b8 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:57:57.073681 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4a8c12-9736-4ea6-96f1-cedec314c9b8 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:57:57.197076 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 24/46] 10.222.0.55 () {60 vars in 1141 bytes} [Fri Sep 23 14:57:56 2022] POST /identity/v3/projects => generated 312 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 23 14:57:58.568400 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d9bc69-47e1-4f06-9603-43cc56733805 None None] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:57:58.568950 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d9bc69-47e1-4f06-9603-43cc56733805 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:57:58.568950 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d9bc69-47e1-4f06-9603-43cc56733805 None None] PATH_INFO: `/` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:57:58.570293 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 23/47] 10.222.0.55 () {54 vars in 927 bytes} [Fri Sep 23 14:57:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 14:57:58.575639 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00486b3-4dd5-48ea-be00-351db070cafb None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:57:58.576113 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00486b3-4dd5-48ea-be00-351db070cafb None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:57:58.576221 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00486b3-4dd5-48ea-be00-351db070cafb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:57:58.594673 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-b00486b3-4dd5-48ea-be00-351db070cafb None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 14:57:58.596188 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-b00486b3-4dd5-48ea-be00-351db070cafb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 14:57:58.662527 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 25/48] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Sep 23 14:57:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 23 14:57:58.928523 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d9bc69-47e1-4f06-9603-43cc56733805 None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:57:59.030491 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d9bc69-47e1-4f06-9603-43cc56733805 None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:57:59.045117 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:57:59.064624 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1693e376-249a-4f20-8b18-5b5322b21aac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:57:59.065271 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1693e376-249a-4f20-8b18-5b5322b21aac None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:57:59.065558 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1693e376-249a-4f20-8b18-5b5322b21aac None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:57:59.065835 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1693e376-249a-4f20-8b18-5b5322b21aac None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:57:59.076714 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1693e376-249a-4f20-8b18-5b5322b21aac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 14:57:59.082118 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application [None req-1693e376-249a-4f20-8b18-5b5322b21aac None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 23 14:57:59.082118 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 14:57:59.082118 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 14:57:59.082118 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 14:57:59.082118 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 14:57:59.082118 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 14:57:59.082118 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 14:57:59.082118 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 14:57:59.082118 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 14:57:59.082118 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 14:57:59.082118 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 14:57:59.082118 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 14:57:59.082118 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 14:57:59.082118 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 14:57:59.082118 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 14:57:59.082118 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 14:57:59.083179 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 14:57:59.083179 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 14:57:59.083179 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 14:57:59.083179 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 14:57:59.083179 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 14:57:59.083179 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 14:57:59.083179 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 14:57:59.083179 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 14:57:59.083179 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 14:57:59.083179 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 14:57:59.083179 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 14:57:59.083179 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application with Lock( Sep 23 14:57:59.083179 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 14:57:59.083179 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._enter() Sep 23 14:57:59.083179 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 14:57:59.083179 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 14:57:59.083945 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 14:57:59.083945 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.creator() Sep 23 14:57:59.083945 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 14:57:59.083945 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application created_value = creator( Sep 23 14:57:59.083945 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 14:57:59.083945 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 14:57:59.083945 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 14:57:59.083945 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 14:57:59.083945 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 14:57:59.083945 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 14:57:59.083945 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 23 14:57:59.083945 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Sep 23 14:57:59.083945 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 24/49] 10.222.0.55 () {56 vars in 1096 bytes} [Fri Sep 23 14:57:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 14:57:59.089745 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00486b3-4dd5-48ea-be00-351db070cafb None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:57:59.129759 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:57:59.190789 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00486b3-4dd5-48ea-be00-351db070cafb None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:57:59.225482 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181e3b93-71db-43db-a20e-eb892bec1ef3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:57:59.226131 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181e3b93-71db-43db-a20e-eb892bec1ef3 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:57:59.226415 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181e3b93-71db-43db-a20e-eb892bec1ef3 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:57:59.226694 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181e3b93-71db-43db-a20e-eb892bec1ef3 None admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:57:59.227394 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181e3b93-71db-43db-a20e-eb892bec1ef3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:57:59.227858 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181e3b93-71db-43db-a20e-eb892bec1ef3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:57:59.229325 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181e3b93-71db-43db-a20e-eb892bec1ef3 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:57:59.237674 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 26/50] 10.222.0.55 () {56 vars in 1099 bytes} [Fri Sep 23 14:57:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:57:59.243635 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1693e376-249a-4f20-8b18-5b5322b21aac None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:57:59.349047 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1693e376-249a-4f20-8b18-5b5322b21aac None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:57:59.382042 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11611aa7-01c3-478e-b222-fd3349137891 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:57:59.382720 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11611aa7-01c3-478e-b222-fd3349137891 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:57:59.383026 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11611aa7-01c3-478e-b222-fd3349137891 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:57:59.383273 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11611aa7-01c3-478e-b222-fd3349137891 None admin] PATH_INFO: `/v3/users/0fc10e0211a54a0faf7ea9b86542de03` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:57:59.385143 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11611aa7-01c3-478e-b222-fd3349137891 None admin] RBAC: Authorizing `identity:get_user(user_id=0fc10e0211a54a0faf7ea9b86542de03)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:57:59.386600 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11611aa7-01c3-478e-b222-fd3349137891 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:57:59.388176 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 25/51] 10.222.0.55 () {56 vars in 1177 bytes} [Fri Sep 23 14:57:59 2022] GET /identity/v3/users/0fc10e0211a54a0faf7ea9b86542de03 => generated 247 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:57:59.393610 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181e3b93-71db-43db-a20e-eb892bec1ef3 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:57:59.422396 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d5aeeb-4d70-432f-af13-4ed49fefe7d4 None None] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:57:59.422689 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d5aeeb-4d70-432f-af13-4ed49fefe7d4 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:57:59.422963 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d5aeeb-4d70-432f-af13-4ed49fefe7d4 None None] PATH_INFO: `/` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:57:59.424333 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 26/52] 10.222.0.55 () {54 vars in 927 bytes} [Fri Sep 23 14:57:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 14:57:59.429661 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe39b9a-2492-4b92-82df-fd13bb31756c None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:57:59.429948 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe39b9a-2492-4b92-82df-fd13bb31756c None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:57:59.430227 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe39b9a-2492-4b92-82df-fd13bb31756c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:57:59.449432 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-bfe39b9a-2492-4b92-82df-fd13bb31756c None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 14:57:59.450538 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-bfe39b9a-2492-4b92-82df-fd13bb31756c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 14:57:59.489255 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:57:59.504034 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:57:59.506920 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181e3b93-71db-43db-a20e-eb892bec1ef3 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:57:59.525092 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 27/53] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Sep 23 14:57:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 23 14:57:59.544307 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba76ff0-cbb1-47d2-a67d-011183b0b61a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:57:59.544948 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba76ff0-cbb1-47d2-a67d-011183b0b61a None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:57:59.545227 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba76ff0-cbb1-47d2-a67d-011183b0b61a None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:57:59.545500 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba76ff0-cbb1-47d2-a67d-011183b0b61a None admin] PATH_INFO: `/v3/domains/default` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:57:59.546730 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba76ff0-cbb1-47d2-a67d-011183b0b61a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:57:59.548203 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba76ff0-cbb1-47d2-a67d-011183b0b61a None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:57:59.549805 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 27/54] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Sep 23 14:57:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:57:59.556359 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe39b9a-2492-4b92-82df-fd13bb31756c None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:57:59.660320 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe39b9a-2492-4b92-82df-fd13bb31756c None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:57:59.695373 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb34c73-0a04-48ae-a26c-e170fdb6874b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:57:59.696027 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb34c73-0a04-48ae-a26c-e170fdb6874b None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:57:59.696297 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb34c73-0a04-48ae-a26c-e170fdb6874b None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:57:59.696613 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb34c73-0a04-48ae-a26c-e170fdb6874b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:57:59.697336 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb34c73-0a04-48ae-a26c-e170fdb6874b None admin] RBAC: Adding query filter params (role.id=8e2919e79df040e8926f21607818b58c, scope.domain.id=default, user.id=0fc10e0211a54a0faf7ea9b86542de03) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:57:59.697737 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb34c73-0a04-48ae-a26c-e170fdb6874b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:57:59.699373 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb34c73-0a04-48ae-a26c-e170fdb6874b None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:57:59.706881 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 28/55] 10.222.0.55 () {56 vars in 1322 bytes} [Fri Sep 23 14:57:59 2022] GET /identity/v3/role_assignments?user.id=0fc10e0211a54a0faf7ea9b86542de03&scope.domain.id=default&role.id=8e2919e79df040e8926f21607818b58c => generated 530 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:57:59.826078 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba76ff0-cbb1-47d2-a67d-011183b0b61a None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:00.059551 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba76ff0-cbb1-47d2-a67d-011183b0b61a None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:00.105563 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e22e58f-0ca3-4dac-8d15-a3a9a4f55918 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:00.106594 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e22e58f-0ca3-4dac-8d15-a3a9a4f55918 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:00.107061 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e22e58f-0ca3-4dac-8d15-a3a9a4f55918 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:00.107514 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e22e58f-0ca3-4dac-8d15-a3a9a4f55918 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:00.110292 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e22e58f-0ca3-4dac-8d15-a3a9a4f55918 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:00.111762 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e22e58f-0ca3-4dac-8d15-a3a9a4f55918 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:00.113379 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 28/56] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Sep 23 14:57:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:58:00.119669 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb34c73-0a04-48ae-a26c-e170fdb6874b None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:00.162949 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:00.243360 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb34c73-0a04-48ae-a26c-e170fdb6874b None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:00.278275 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6850c41-a36f-4094-ac5d-7db224a71033 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:00.278881 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6850c41-a36f-4094-ac5d-7db224a71033 None admin] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:00.279170 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6850c41-a36f-4094-ac5d-7db224a71033 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:00.279454 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6850c41-a36f-4094-ac5d-7db224a71033 None admin] PATH_INFO: `/v3/projects` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:00.280149 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6850c41-a36f-4094-ac5d-7db224a71033 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:00.281587 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6850c41-a36f-4094-ac5d-7db224a71033 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:00.435399 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 29/57] 10.222.0.55 () {60 vars in 1140 bytes} [Fri Sep 23 14:58:00 2022] POST /identity/v3/projects => generated 298 bytes in 312 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 14:58:02.610521 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0797ba1b-6ad6-4a50-808c-81b9b61ab3b3 None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:02.611067 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0797ba1b-6ad6-4a50-808c-81b9b61ab3b3 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:02.611067 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0797ba1b-6ad6-4a50-808c-81b9b61ab3b3 None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:02.612463 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 29/58] 10.222.0.55 () {54 vars in 927 bytes} [Fri Sep 23 14:58:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 14:58:02.617666 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd27681-02db-4be1-bd19-48d2cf74930f None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:02.618072 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd27681-02db-4be1-bd19-48d2cf74930f None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:02.618228 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd27681-02db-4be1-bd19-48d2cf74930f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:02.638116 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-6cd27681-02db-4be1-bd19-48d2cf74930f None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 14:58:02.639315 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-6cd27681-02db-4be1-bd19-48d2cf74930f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 14:58:02.708316 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 30/59] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Sep 23 14:58:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 14:58:02.981929 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0797ba1b-6ad6-4a50-808c-81b9b61ab3b3 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:03.018614 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:03.086340 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0797ba1b-6ad6-4a50-808c-81b9b61ab3b3 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:03.121602 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e705ea-7aa9-4ca4-91b6-ccc84c98c79c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:03.122233 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e705ea-7aa9-4ca4-91b6-ccc84c98c79c None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:03.122520 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e705ea-7aa9-4ca4-91b6-ccc84c98c79c None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:03.122801 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e705ea-7aa9-4ca4-91b6-ccc84c98c79c None admin] PATH_INFO: `/v3/domains/default` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:03.123967 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e705ea-7aa9-4ca4-91b6-ccc84c98c79c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:03.125430 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e705ea-7aa9-4ca4-91b6-ccc84c98c79c None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:03.126997 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 30/60] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Sep 23 14:58:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:58:03.132656 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd27681-02db-4be1-bd19-48d2cf74930f None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:03.151346 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:03.231563 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd27681-02db-4be1-bd19-48d2cf74930f None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:03.265703 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c846f676-392d-4c81-9f49-751ff7cceb13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:03.266248 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c846f676-392d-4c81-9f49-751ff7cceb13 None admin] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:03.266531 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c846f676-392d-4c81-9f49-751ff7cceb13 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:03.266808 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c846f676-392d-4c81-9f49-751ff7cceb13 None admin] PATH_INFO: `/v3/users` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:03.267465 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c846f676-392d-4c81-9f49-751ff7cceb13 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:03.268953 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c846f676-392d-4c81-9f49-751ff7cceb13 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:03.410896 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 31/61] 10.222.0.55 () {60 vars in 1132 bytes} [Fri Sep 23 14:58:03 2022] POST /identity/v3/users => generated 275 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 14:58:06.662915 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e245869-76fb-465d-af3e-6f2575f7ae9d None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:06.663464 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e245869-76fb-465d-af3e-6f2575f7ae9d None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:06.663464 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e245869-76fb-465d-af3e-6f2575f7ae9d None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:06.666620 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 31/62] 10.222.0.55 () {54 vars in 927 bytes} [Fri Sep 23 14:58:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 14:58:06.673423 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17574701-8532-4064-9859-1cd48bd988ec None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:06.673857 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17574701-8532-4064-9859-1cd48bd988ec None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:06.674067 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17574701-8532-4064-9859-1cd48bd988ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:06.702460 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-17574701-8532-4064-9859-1cd48bd988ec None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 14:58:06.703746 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-17574701-8532-4064-9859-1cd48bd988ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 14:58:06.760152 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7415d8f-115b-4c9b-8458-01495f1ded00 None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:06.760791 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7415d8f-115b-4c9b-8458-01495f1ded00 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:06.761268 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7415d8f-115b-4c9b-8458-01495f1ded00 None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:06.762830 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 32/63] 10.222.0.55 () {54 vars in 927 bytes} [Fri Sep 23 14:58:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 23 14:58:06.769439 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a45d48-8f4b-43eb-9c44-713501275e01 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:06.770003 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a45d48-8f4b-43eb-9c44-713501275e01 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:06.770477 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a45d48-8f4b-43eb-9c44-713501275e01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:06.791937 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-80a45d48-8f4b-43eb-9c44-713501275e01 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 14:58:06.793545 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-80a45d48-8f4b-43eb-9c44-713501275e01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 14:58:06.806013 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 32/64] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Sep 23 14:58:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 23 14:58:06.887700 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 33/65] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Sep 23 14:58:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 23 14:58:07.165871 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17574701-8532-4064-9859-1cd48bd988ec None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:07.169536 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:07.231248 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a45d48-8f4b-43eb-9c44-713501275e01 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:07.291963 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:07.365383 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17574701-8532-4064-9859-1cd48bd988ec None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:07.432577 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4f8bd4-6cf8-4091-b870-62c5636cb8c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:07.433238 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4f8bd4-6cf8-4091-b870-62c5636cb8c0 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:07.433523 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4f8bd4-6cf8-4091-b870-62c5636cb8c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:07.434456 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4f8bd4-6cf8-4091-b870-62c5636cb8c0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:07.438700 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a45d48-8f4b-43eb-9c44-713501275e01 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:07.456713 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4f8bd4-6cf8-4091-b870-62c5636cb8c0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 14:58:07.462723 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application [None req-6c4f8bd4-6cf8-4091-b870-62c5636cb8c0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 23 14:58:07.462723 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 14:58:07.462723 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 14:58:07.462723 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 14:58:07.462723 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 14:58:07.462723 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 14:58:07.462723 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 14:58:07.462723 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 14:58:07.462723 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 14:58:07.462723 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 14:58:07.462723 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 14:58:07.462723 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 14:58:07.462723 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 14:58:07.462723 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 14:58:07.462723 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 14:58:07.462723 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 14:58:07.463828 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 14:58:07.463828 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 14:58:07.463828 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 14:58:07.463828 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 14:58:07.463828 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 14:58:07.463828 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 14:58:07.463828 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 14:58:07.463828 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 14:58:07.463828 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 14:58:07.463828 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 14:58:07.463828 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 14:58:07.463828 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application with Lock( Sep 23 14:58:07.463828 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 14:58:07.463828 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._enter() Sep 23 14:58:07.463828 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 14:58:07.463828 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 14:58:07.464859 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 14:58:07.464859 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.creator() Sep 23 14:58:07.464859 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 14:58:07.464859 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application created_value = creator( Sep 23 14:58:07.464859 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 14:58:07.464859 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 14:58:07.464859 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 14:58:07.464859 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 14:58:07.464859 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 14:58:07.464859 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 14:58:07.464859 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 23 14:58:07.464859 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Sep 23 14:58:07.467103 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 33/66] 10.222.0.55 () {56 vars in 1096 bytes} [Fri Sep 23 14:58:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 14:58:07.469336 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f72272a-ba65-4d0d-9c6b-61f75aaeb931 None None] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:07.469874 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f72272a-ba65-4d0d-9c6b-61f75aaeb931 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:07.470307 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f72272a-ba65-4d0d-9c6b-61f75aaeb931 None None] PATH_INFO: `/` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:07.471940 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 34/67] 10.222.0.55 () {54 vars in 927 bytes} [Fri Sep 23 14:58:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 14:58:07.473875 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f72272a-ba65-4d0d-9c6b-61f75aaeb931 None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:07.495773 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839b745a-f09c-4de3-8c4c-3763530fa646 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:07.496488 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839b745a-f09c-4de3-8c4c-3763530fa646 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:07.496792 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839b745a-f09c-4de3-8c4c-3763530fa646 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:07.497070 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839b745a-f09c-4de3-8c4c-3763530fa646 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:07.508480 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839b745a-f09c-4de3-8c4c-3763530fa646 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 14:58:07.513860 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-839b745a-f09c-4de3-8c4c-3763530fa646 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 23 14:58:07.513860 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 14:58:07.513860 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 14:58:07.513860 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 14:58:07.513860 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 14:58:07.513860 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 14:58:07.513860 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 14:58:07.513860 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 14:58:07.513860 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 14:58:07.513860 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 14:58:07.513860 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 14:58:07.513860 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 14:58:07.513860 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 14:58:07.513860 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 14:58:07.513860 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 14:58:07.513860 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 14:58:07.514736 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 14:58:07.514736 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 14:58:07.514736 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 14:58:07.514736 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 14:58:07.514736 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 14:58:07.514736 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 14:58:07.514736 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 14:58:07.514736 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 14:58:07.514736 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 14:58:07.514736 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 14:58:07.514736 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 14:58:07.514736 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Sep 23 14:58:07.514736 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 14:58:07.514736 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Sep 23 14:58:07.514736 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 14:58:07.514736 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 14:58:07.515573 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 14:58:07.515573 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Sep 23 14:58:07.515573 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 14:58:07.515573 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Sep 23 14:58:07.515573 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 14:58:07.515573 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 14:58:07.515573 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 14:58:07.515573 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 14:58:07.515573 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 14:58:07.515573 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 14:58:07.515573 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 23 14:58:07.515573 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Sep 23 14:58:07.515573 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 34/68] 10.222.0.55 () {56 vars in 1099 bytes} [Fri Sep 23 14:58:07 2022] GET /identity/v3/roles/member => generated 84 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 23 14:58:07.521591 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5c700f-6c49-4be0-bf87-ad90e449e8c0 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:07.521872 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5c700f-6c49-4be0-bf87-ad90e449e8c0 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:07.522249 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5c700f-6c49-4be0-bf87-ad90e449e8c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:07.540571 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-fd5c700f-6c49-4be0-bf87-ad90e449e8c0 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 14:58:07.541756 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-fd5c700f-6c49-4be0-bf87-ad90e449e8c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 14:58:07.547289 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:07.591198 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f72272a-ba65-4d0d-9c6b-61f75aaeb931 None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:07.615158 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 35/69] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Sep 23 14:58:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 14:58:07.616532 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1775e73d-36cc-4481-9bf8-41bbd775e48c None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:07.616779 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1775e73d-36cc-4481-9bf8-41bbd775e48c None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:07.617039 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1775e73d-36cc-4481-9bf8-41bbd775e48c None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:07.618625 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 36/70] 10.222.0.55 () {54 vars in 927 bytes} [Fri Sep 23 14:58:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 14:58:07.619253 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1775e73d-36cc-4481-9bf8-41bbd775e48c None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:07.626457 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6147f0b8-cb16-4f9e-93f1-b2d39d6e1c94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:07.627070 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6147f0b8-cb16-4f9e-93f1-b2d39d6e1c94 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:07.627350 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6147f0b8-cb16-4f9e-93f1-b2d39d6e1c94 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:07.627620 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6147f0b8-cb16-4f9e-93f1-b2d39d6e1c94 None admin] PATH_INFO: `/v3/roles` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:07.628322 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6147f0b8-cb16-4f9e-93f1-b2d39d6e1c94 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:58:07.628620 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6147f0b8-cb16-4f9e-93f1-b2d39d6e1c94 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:07.630001 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6147f0b8-cb16-4f9e-93f1-b2d39d6e1c94 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:07.647137 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 35/71] 10.222.0.55 () {56 vars in 1099 bytes} [Fri Sep 23 14:58:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:58:07.647137 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92defb63-d4f6-467f-9076-96e3204e7c0a None None] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:07.647137 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92defb63-d4f6-467f-9076-96e3204e7c0a None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:07.647137 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92defb63-d4f6-467f-9076-96e3204e7c0a None None] PATH_INFO: `/` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:07.647137 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 36/72] 10.222.0.55 () {54 vars in 927 bytes} [Fri Sep 23 14:58:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 14:58:07.647137 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18ef138-9519-4d53-a9ec-ccba9c738e3d None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:07.647612 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18ef138-9519-4d53-a9ec-ccba9c738e3d None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:07.647612 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18ef138-9519-4d53-a9ec-ccba9c738e3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:07.664037 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:07.667455 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-c18ef138-9519-4d53-a9ec-ccba9c738e3d None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 14:58:07.668759 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-c18ef138-9519-4d53-a9ec-ccba9c738e3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 14:58:07.744623 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 37/73] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Sep 23 14:58:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 14:58:07.748917 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18ef138-9519-4d53-a9ec-ccba9c738e3d None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:07.752440 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1775e73d-36cc-4481-9bf8-41bbd775e48c None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:07.800574 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cf9ec8-d070-48cb-a366-20845323487f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:07.801684 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cf9ec8-d070-48cb-a366-20845323487f None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:07.802142 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cf9ec8-d070-48cb-a366-20845323487f None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:07.802588 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cf9ec8-d070-48cb-a366-20845323487f None admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:07.803531 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25cf9ec8-d070-48cb-a366-20845323487f None admin] RBAC: Adding query filter params (name=member) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:58:07.804102 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25cf9ec8-d070-48cb-a366-20845323487f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:07.805954 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25cf9ec8-d070-48cb-a366-20845323487f None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:07.816607 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 37/74] 10.222.0.55 () {56 vars in 1101 bytes} [Fri Sep 23 14:58:07 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:58:07.818597 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fb633f-9331-448a-8985-854900fcf467 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:07.819683 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:07.820621 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fb633f-9331-448a-8985-854900fcf467 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:07.821314 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fb633f-9331-448a-8985-854900fcf467 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:07.842803 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-b2fb633f-9331-448a-8985-854900fcf467 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 14:58:07.844323 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-b2fb633f-9331-448a-8985-854900fcf467 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 14:58:07.876551 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18ef138-9519-4d53-a9ec-ccba9c738e3d None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:07.913565 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f31c575-592b-4126-8fb3-971e920c76e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:07.914508 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f31c575-592b-4126-8fb3-971e920c76e0 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:07.914957 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f31c575-592b-4126-8fb3-971e920c76e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:07.916078 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f31c575-592b-4126-8fb3-971e920c76e0 None admin] PATH_INFO: `/v3/users/0fc10e0211a54a0faf7ea9b86542de03` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:07.917571 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f31c575-592b-4126-8fb3-971e920c76e0 None admin] RBAC: Authorizing `identity:get_user(user_id=0fc10e0211a54a0faf7ea9b86542de03)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:07.919341 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f31c575-592b-4126-8fb3-971e920c76e0 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:07.921172 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 38/75] 10.222.0.55 () {56 vars in 1177 bytes} [Fri Sep 23 14:58:07 2022] GET /identity/v3/users/0fc10e0211a54a0faf7ea9b86542de03 => generated 247 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:58:07.922183 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f31c575-592b-4126-8fb3-971e920c76e0 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:07.926900 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 38/76] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Sep 23 14:58:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 14:58:07.927961 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fb633f-9331-448a-8985-854900fcf467 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:07.946873 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:08.038458 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f31c575-592b-4126-8fb3-971e920c76e0 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:08.052958 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fb633f-9331-448a-8985-854900fcf467 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:08.068287 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:08.082175 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9cc5b5-c7b1-41a6-83de-6a33efdd40bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:08.083011 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9cc5b5-c7b1-41a6-83de-6a33efdd40bf None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:08.083365 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9cc5b5-c7b1-41a6-83de-6a33efdd40bf None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:08.083657 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9cc5b5-c7b1-41a6-83de-6a33efdd40bf None admin] PATH_INFO: `/v3/users/ca9091e894bc4e01ab24bbbb474a09cd` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:08.096159 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c6386a-c9db-46ea-a4ff-5d7221a85d64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:08.097015 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c6386a-c9db-46ea-a4ff-5d7221a85d64 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:08.097711 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c6386a-c9db-46ea-a4ff-5d7221a85d64 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:08.097993 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c6386a-c9db-46ea-a4ff-5d7221a85d64 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:08.098957 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9cc5b5-c7b1-41a6-83de-6a33efdd40bf None admin] RBAC: Authorizing `identity:get_user(user_id=ca9091e894bc4e01ab24bbbb474a09cd)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:08.100935 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9cc5b5-c7b1-41a6-83de-6a33efdd40bf None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:08.103427 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 39/77] 10.222.0.55 () {56 vars in 1177 bytes} [Fri Sep 23 14:58:07 2022] GET /identity/v3/users/ca9091e894bc4e01ab24bbbb474a09cd => generated 275 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:58:08.104671 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9cc5b5-c7b1-41a6-83de-6a33efdd40bf None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:08.114903 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c6386a-c9db-46ea-a4ff-5d7221a85d64 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 14:58:08.121683 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-d9c6386a-c9db-46ea-a4ff-5d7221a85d64 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 23 14:58:08.121683 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 14:58:08.121683 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 14:58:08.121683 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 14:58:08.121683 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 14:58:08.121683 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 14:58:08.121683 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 14:58:08.121683 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 14:58:08.121683 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 14:58:08.121683 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 14:58:08.121683 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 14:58:08.121683 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 14:58:08.121683 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 14:58:08.121683 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 14:58:08.121683 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 14:58:08.121683 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 14:58:08.122719 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 14:58:08.122719 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 14:58:08.122719 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 14:58:08.122719 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 14:58:08.122719 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 14:58:08.122719 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 14:58:08.122719 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 14:58:08.122719 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 14:58:08.122719 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 14:58:08.122719 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 14:58:08.122719 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 14:58:08.122719 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Sep 23 14:58:08.122719 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 14:58:08.122719 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Sep 23 14:58:08.122719 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 14:58:08.122719 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 14:58:08.123647 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 14:58:08.123647 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Sep 23 14:58:08.123647 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 14:58:08.123647 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Sep 23 14:58:08.123647 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 14:58:08.123647 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 14:58:08.123647 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 14:58:08.123647 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 14:58:08.123647 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 14:58:08.123647 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 14:58:08.123647 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 23 14:58:08.123647 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Sep 23 14:58:08.123647 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 39/78] 10.222.0.55 () {56 vars in 1114 bytes} [Fri Sep 23 14:58:07 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 14:58:08.123647 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c6386a-c9db-46ea-a4ff-5d7221a85d64 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:08.208612 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:08.213833 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9cc5b5-c7b1-41a6-83de-6a33efdd40bf None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:08.233094 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c6386a-c9db-46ea-a4ff-5d7221a85d64 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:08.248592 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b5bbac-a15b-4da3-99b4-71da41f9ea83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:08.249210 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b5bbac-a15b-4da3-99b4-71da41f9ea83 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:08.249495 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b5bbac-a15b-4da3-99b4-71da41f9ea83 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:08.249769 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b5bbac-a15b-4da3-99b4-71da41f9ea83 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:08.268505 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c820e52-af0c-4a16-b0a6-5acf8b5aeb23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:08.269142 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c820e52-af0c-4a16-b0a6-5acf8b5aeb23 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:08.269426 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c820e52-af0c-4a16-b0a6-5acf8b5aeb23 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:08.269703 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c820e52-af0c-4a16-b0a6-5acf8b5aeb23 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:08.270841 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c820e52-af0c-4a16-b0a6-5acf8b5aeb23 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:08.272406 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c820e52-af0c-4a16-b0a6-5acf8b5aeb23 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:08.274059 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 40/79] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Sep 23 14:58:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:58:08.274979 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c820e52-af0c-4a16-b0a6-5acf8b5aeb23 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:08.301260 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:08.320516 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b5bbac-a15b-4da3-99b4-71da41f9ea83 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:08.322621 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b5bbac-a15b-4da3-99b4-71da41f9ea83 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:08.384109 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c820e52-af0c-4a16-b0a6-5acf8b5aeb23 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:08.419641 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f7a2d9-a50b-4348-964c-30703392a434 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:08.420290 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f7a2d9-a50b-4348-964c-30703392a434 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:08.420573 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f7a2d9-a50b-4348-964c-30703392a434 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:08.420851 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f7a2d9-a50b-4348-964c-30703392a434 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:08.424965 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 40/80] 10.222.0.55 () {58 vars in 1282 bytes} [Fri Sep 23 14:58:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 23 14:58:08.426015 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b5bbac-a15b-4da3-99b4-71da41f9ea83 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:08.436418 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:08.490484 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f7a2d9-a50b-4348-964c-30703392a434 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:08.491966 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f7a2d9-a50b-4348-964c-30703392a434 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:08.509674 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:08.528267 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b5bbac-a15b-4da3-99b4-71da41f9ea83 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:08.563601 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc95fb4-8568-4f42-a4fa-dc46d45f250e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:08.564237 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc95fb4-8568-4f42-a4fa-dc46d45f250e None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:08.564523 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc95fb4-8568-4f42-a4fa-dc46d45f250e None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:08.564802 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc95fb4-8568-4f42-a4fa-dc46d45f250e None admin] PATH_INFO: `/v3/roles` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:08.565466 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc95fb4-8568-4f42-a4fa-dc46d45f250e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:58:08.565773 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc95fb4-8568-4f42-a4fa-dc46d45f250e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:08.567173 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc95fb4-8568-4f42-a4fa-dc46d45f250e None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:08.575349 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 41/81] 10.222.0.55 () {56 vars in 1111 bytes} [Fri Sep 23 14:58:08 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:58:08.576178 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc95fb4-8568-4f42-a4fa-dc46d45f250e None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:08.599994 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 41/82] 10.222.0.55 () {58 vars in 1282 bytes} [Fri Sep 23 14:58:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 23 14:58:08.600838 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f7a2d9-a50b-4348-964c-30703392a434 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:08.646873 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:08.682246 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc95fb4-8568-4f42-a4fa-dc46d45f250e None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:08.703522 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f7a2d9-a50b-4348-964c-30703392a434 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:08.716751 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c3fc46-50dc-4e6b-8733-8c81bd63980c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:08.717391 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c3fc46-50dc-4e6b-8733-8c81bd63980c None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:08.717674 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c3fc46-50dc-4e6b-8733-8c81bd63980c None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:08.717952 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c3fc46-50dc-4e6b-8733-8c81bd63980c None admin] PATH_INFO: `/v3/roles/member` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:08.728475 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c3fc46-50dc-4e6b-8733-8c81bd63980c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 14:58:08.733716 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application [None req-58c3fc46-50dc-4e6b-8733-8c81bd63980c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 23 14:58:08.733716 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 14:58:08.733716 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 14:58:08.733716 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 14:58:08.733716 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 14:58:08.733716 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 14:58:08.733716 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 14:58:08.733716 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 14:58:08.733716 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 14:58:08.733716 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 14:58:08.733716 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 14:58:08.733716 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 14:58:08.733716 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 14:58:08.733716 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 14:58:08.733716 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 14:58:08.733716 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 14:58:08.734664 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 14:58:08.734664 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 14:58:08.734664 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 14:58:08.734664 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 14:58:08.734664 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 14:58:08.734664 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 14:58:08.734664 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 14:58:08.734664 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 14:58:08.734664 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 14:58:08.734664 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 14:58:08.734664 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 14:58:08.734664 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application with Lock( Sep 23 14:58:08.734664 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 14:58:08.734664 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._enter() Sep 23 14:58:08.734664 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 14:58:08.734664 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 14:58:08.735447 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 14:58:08.735447 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.creator() Sep 23 14:58:08.735447 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 14:58:08.735447 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application created_value = creator( Sep 23 14:58:08.735447 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 14:58:08.735447 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 14:58:08.735447 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 14:58:08.735447 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 14:58:08.735447 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 14:58:08.735447 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 14:58:08.735447 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 23 14:58:08.735447 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Sep 23 14:58:08.735447 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 42/83] 10.222.0.55 () {56 vars in 1099 bytes} [Fri Sep 23 14:58:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 14:58:08.736186 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c3fc46-50dc-4e6b-8733-8c81bd63980c None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:08.739084 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:08.743599 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c64587-6e92-402c-a9c6-af13e7bb29d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:08.744154 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c64587-6e92-402c-a9c6-af13e7bb29d8 None admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:08.744361 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c64587-6e92-402c-a9c6-af13e7bb29d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:08.744729 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c64587-6e92-402c-a9c6-af13e7bb29d8 None admin] PATH_INFO: `/v3/users` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:08.745393 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c64587-6e92-402c-a9c6-af13e7bb29d8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:08.746849 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c64587-6e92-402c-a9c6-af13e7bb29d8 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:08.835002 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 42/84] 10.222.0.55 () {60 vars in 1132 bytes} [Fri Sep 23 14:58:08 2022] POST /identity/v3/users => generated 289 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 14:58:08.835381 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c3fc46-50dc-4e6b-8733-8c81bd63980c None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:08.836331 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c64587-6e92-402c-a9c6-af13e7bb29d8 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:08.873836 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c745d815-2cc8-47d1-b184-9708f537cdde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:08.874518 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c745d815-2cc8-47d1-b184-9708f537cdde None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:08.874803 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c745d815-2cc8-47d1-b184-9708f537cdde None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:08.875080 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c745d815-2cc8-47d1-b184-9708f537cdde None admin] PATH_INFO: `/v3/projects/0ffbe3ca86954853be08ecf627d6af00` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:08.876388 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c745d815-2cc8-47d1-b184-9708f537cdde None admin] RBAC: Authorizing `identity:get_project(project_id=0ffbe3ca86954853be08ecf627d6af00)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:08.877947 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c745d815-2cc8-47d1-b184-9708f537cdde None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:08.884772 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 43/85] 10.222.0.55 () {56 vars in 1186 bytes} [Fri Sep 23 14:58:08 2022] GET /identity/v3/projects/0ffbe3ca86954853be08ecf627d6af00 => generated 298 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:58:08.884772 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c745d815-2cc8-47d1-b184-9708f537cdde None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:08.953166 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:08.956338 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c64587-6e92-402c-a9c6-af13e7bb29d8 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:08.965389 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:08.996422 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a21fc7-cd82-472a-a7be-b54c99e57c30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:08.997183 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a21fc7-cd82-472a-a7be-b54c99e57c30 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:08.997809 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a21fc7-cd82-472a-a7be-b54c99e57c30 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:08.998184 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a21fc7-cd82-472a-a7be-b54c99e57c30 None admin] PATH_INFO: `/v3/users/ca9091e894bc4e01ab24bbbb474a09cd` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:08.999438 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a21fc7-cd82-472a-a7be-b54c99e57c30 None admin] RBAC: Authorizing `identity:get_user(user_id=ca9091e894bc4e01ab24bbbb474a09cd)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:09.000802 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c745d815-2cc8-47d1-b184-9708f537cdde None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:09.000953 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a21fc7-cd82-472a-a7be-b54c99e57c30 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:09.002611 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 43/86] 10.222.0.55 () {56 vars in 1177 bytes} [Fri Sep 23 14:58:08 2022] GET /identity/v3/users/ca9091e894bc4e01ab24bbbb474a09cd => generated 275 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:58:09.003479 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a21fc7-cd82-472a-a7be-b54c99e57c30 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:09.044790 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b479917f-2d35-4062-8394-1abfb56238b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:09.045447 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b479917f-2d35-4062-8394-1abfb56238b1 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:09.045734 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b479917f-2d35-4062-8394-1abfb56238b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:09.046049 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b479917f-2d35-4062-8394-1abfb56238b1 None admin] PATH_INFO: `/v3/projects/0ffbe3ca86954853be08ecf627d6af00` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:09.047174 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b479917f-2d35-4062-8394-1abfb56238b1 None admin] RBAC: Authorizing `identity:get_project(project_id=0ffbe3ca86954853be08ecf627d6af00)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:09.048732 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b479917f-2d35-4062-8394-1abfb56238b1 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:09.052024 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 44/87] 10.222.0.55 () {56 vars in 1186 bytes} [Fri Sep 23 14:58:08 2022] GET /identity/v3/projects/0ffbe3ca86954853be08ecf627d6af00 => generated 298 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:58:09.052024 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b479917f-2d35-4062-8394-1abfb56238b1 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:09.139751 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a21fc7-cd82-472a-a7be-b54c99e57c30 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:09.181589 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b479917f-2d35-4062-8394-1abfb56238b1 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:09.183446 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727e6cd6-1767-45ba-96c0-1a4982e30b13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:09.184115 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727e6cd6-1767-45ba-96c0-1a4982e30b13 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:09.184401 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727e6cd6-1767-45ba-96c0-1a4982e30b13 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:09.184735 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727e6cd6-1767-45ba-96c0-1a4982e30b13 None admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:09.185443 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727e6cd6-1767-45ba-96c0-1a4982e30b13 None admin] RBAC: Adding query filter params (name=member) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:58:09.185749 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727e6cd6-1767-45ba-96c0-1a4982e30b13 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:09.187143 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727e6cd6-1767-45ba-96c0-1a4982e30b13 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:09.197325 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 44/88] 10.222.0.55 () {56 vars in 1101 bytes} [Fri Sep 23 14:58:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:58:09.198170 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727e6cd6-1767-45ba-96c0-1a4982e30b13 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:09.228368 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d725fc37-249e-40c6-96a2-364c088a5f21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:09.229000 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d725fc37-249e-40c6-96a2-364c088a5f21 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:09.229290 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d725fc37-249e-40c6-96a2-364c088a5f21 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:09.230395 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d725fc37-249e-40c6-96a2-364c088a5f21 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:09.230395 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d725fc37-249e-40c6-96a2-364c088a5f21 None admin] RBAC: Adding query filter params (role.id=8e2919e79df040e8926f21607818b58c, scope.project.id=0ffbe3ca86954853be08ecf627d6af00, user.id=0fc10e0211a54a0faf7ea9b86542de03) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:58:09.230570 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d725fc37-249e-40c6-96a2-364c088a5f21 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:09.232085 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d725fc37-249e-40c6-96a2-364c088a5f21 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:09.234579 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:09.238627 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 45/89] 10.222.0.55 () {56 vars in 1374 bytes} [Fri Sep 23 14:58:09 2022] GET /identity/v3/role_assignments?user.id=0fc10e0211a54a0faf7ea9b86542de03&scope.project.id=0ffbe3ca86954853be08ecf627d6af00&role.id=8e2919e79df040e8926f21607818b58c => generated 259 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:58:09.239293 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d725fc37-249e-40c6-96a2-364c088a5f21 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:09.297313 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:09.314799 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727e6cd6-1767-45ba-96c0-1a4982e30b13 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:09.353274 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d889e3ae-89b1-4ba6-98d0-924dc2052f04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:09.353919 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d889e3ae-89b1-4ba6-98d0-924dc2052f04 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:09.354203 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d889e3ae-89b1-4ba6-98d0-924dc2052f04 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:09.354303 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d725fc37-249e-40c6-96a2-364c088a5f21 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:09.354486 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d889e3ae-89b1-4ba6-98d0-924dc2052f04 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:09.402259 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604eea87-0311-43c4-9937-8d04dab82e1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:09.402896 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604eea87-0311-43c4-9937-8d04dab82e1c None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:09.403187 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604eea87-0311-43c4-9937-8d04dab82e1c None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:09.403465 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604eea87-0311-43c4-9937-8d04dab82e1c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:09.404230 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604eea87-0311-43c4-9937-8d04dab82e1c None admin] RBAC: Adding query filter params (role.id=af257be2dee54d6b942f58650196582c, scope.project.id=0ffbe3ca86954853be08ecf627d6af00, user.id=ca9091e894bc4e01ab24bbbb474a09cd) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:58:09.404543 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604eea87-0311-43c4-9937-8d04dab82e1c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:09.406022 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604eea87-0311-43c4-9937-8d04dab82e1c None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:09.420768 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 46/90] 10.222.0.55 () {56 vars in 1374 bytes} [Fri Sep 23 14:58:09 2022] GET /identity/v3/role_assignments?user.id=ca9091e894bc4e01ab24bbbb474a09cd&scope.project.id=0ffbe3ca86954853be08ecf627d6af00&role.id=af257be2dee54d6b942f58650196582c => generated 259 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:58:09.421635 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604eea87-0311-43c4-9937-8d04dab82e1c None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:09.433526 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d889e3ae-89b1-4ba6-98d0-924dc2052f04 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:09.435161 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d889e3ae-89b1-4ba6-98d0-924dc2052f04 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:09.473592 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:09.558616 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 45/91] 10.222.0.55 () {58 vars in 1282 bytes} [Fri Sep 23 14:58:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 23 14:58:09.569167 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d889e3ae-89b1-4ba6-98d0-924dc2052f04 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:09.585588 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:09.644580 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604eea87-0311-43c4-9937-8d04dab82e1c None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:09.691657 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfbb0aa-dfb6-452c-8251-4d5ebff0da89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:09.698339 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfbb0aa-dfb6-452c-8251-4d5ebff0da89 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:09.698339 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfbb0aa-dfb6-452c-8251-4d5ebff0da89 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:09.698339 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfbb0aa-dfb6-452c-8251-4d5ebff0da89 None admin] PATH_INFO: `/v3/users/ca9091e894bc4e01ab24bbbb474a09cd` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:09.698339 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfbb0aa-dfb6-452c-8251-4d5ebff0da89 None admin] RBAC: Authorizing `identity:get_user(user_id=ca9091e894bc4e01ab24bbbb474a09cd)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:09.698339 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfbb0aa-dfb6-452c-8251-4d5ebff0da89 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:09.700271 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 47/92] 10.222.0.55 () {56 vars in 1177 bytes} [Fri Sep 23 14:58:09 2022] GET /identity/v3/users/ca9091e894bc4e01ab24bbbb474a09cd => generated 275 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:58:09.705897 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfbb0aa-dfb6-452c-8251-4d5ebff0da89 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:09.727048 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d889e3ae-89b1-4ba6-98d0-924dc2052f04 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:09.820735 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:09.831675 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4ccee2-beed-47b3-8900-34506147f642 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:09.832354 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4ccee2-beed-47b3-8900-34506147f642 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:09.832640 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4ccee2-beed-47b3-8900-34506147f642 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:09.835754 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4ccee2-beed-47b3-8900-34506147f642 None admin] PATH_INFO: `/v3/projects/0ffbe3ca86954853be08ecf627d6af00` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:09.835754 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4ccee2-beed-47b3-8900-34506147f642 None admin] RBAC: Authorizing `identity:get_project(project_id=0ffbe3ca86954853be08ecf627d6af00)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:09.836335 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4ccee2-beed-47b3-8900-34506147f642 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:09.839046 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 46/93] 10.222.0.55 () {56 vars in 1186 bytes} [Fri Sep 23 14:58:09 2022] GET /identity/v3/projects/0ffbe3ca86954853be08ecf627d6af00 => generated 298 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:58:09.846687 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4ccee2-beed-47b3-8900-34506147f642 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:09.872152 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfbb0aa-dfb6-452c-8251-4d5ebff0da89 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:09.956151 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be427f5-5afe-49f8-8d8a-a1455eee52c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:09.956785 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be427f5-5afe-49f8-8d8a-a1455eee52c2 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:09.957071 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be427f5-5afe-49f8-8d8a-a1455eee52c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:09.957347 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be427f5-5afe-49f8-8d8a-a1455eee52c2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:09.980361 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:10.084177 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4ccee2-beed-47b3-8900-34506147f642 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:10.124002 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be427f5-5afe-49f8-8d8a-a1455eee52c2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:10.125679 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be427f5-5afe-49f8-8d8a-a1455eee52c2 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:10.154064 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47660014-cfef-4e62-9f79-31f958f007fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:10.154749 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47660014-cfef-4e62-9f79-31f958f007fd None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:10.155034 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47660014-cfef-4e62-9f79-31f958f007fd None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:10.155344 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47660014-cfef-4e62-9f79-31f958f007fd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:10.156165 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47660014-cfef-4e62-9f79-31f958f007fd None admin] RBAC: Adding query filter params (role.id=5c74fdc0e9cc44538cd8282006ed4443, scope.project.id=0ffbe3ca86954853be08ecf627d6af00, user.id=ca9091e894bc4e01ab24bbbb474a09cd) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:58:10.156479 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47660014-cfef-4e62-9f79-31f958f007fd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:10.158191 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47660014-cfef-4e62-9f79-31f958f007fd None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:10.169343 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 47/94] 10.222.0.55 () {56 vars in 1374 bytes} [Fri Sep 23 14:58:09 2022] GET /identity/v3/role_assignments?user.id=ca9091e894bc4e01ab24bbbb474a09cd&scope.project.id=0ffbe3ca86954853be08ecf627d6af00&role.id=5c74fdc0e9cc44538cd8282006ed4443 => generated 259 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:58:10.282411 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 48/95] 10.222.0.55 () {58 vars in 1282 bytes} [Fri Sep 23 14:58:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 577 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 23 14:58:10.288275 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47660014-cfef-4e62-9f79-31f958f007fd None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:10.373803 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:10.423717 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47660014-cfef-4e62-9f79-31f958f007fd None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:10.474266 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c560d5a9-6e05-4b56-a9bb-16c3af4ddc81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:10.474926 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c560d5a9-6e05-4b56-a9bb-16c3af4ddc81 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:10.475214 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c560d5a9-6e05-4b56-a9bb-16c3af4ddc81 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:10.475496 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c560d5a9-6e05-4b56-a9bb-16c3af4ddc81 None admin] PATH_INFO: `/v3/projects/8381c67dd5274d1780bb20df47b105ec` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:10.480793 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c560d5a9-6e05-4b56-a9bb-16c3af4ddc81 None admin] RBAC: Authorizing `identity:get_project(project_id=8381c67dd5274d1780bb20df47b105ec)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:10.482283 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c560d5a9-6e05-4b56-a9bb-16c3af4ddc81 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:10.484233 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 48/96] 10.222.0.55 () {56 vars in 1186 bytes} [Fri Sep 23 14:58:10 2022] GET /identity/v3/projects/8381c67dd5274d1780bb20df47b105ec => generated 312 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:58:10.490478 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be427f5-5afe-49f8-8d8a-a1455eee52c2 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:10.530758 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:10.623006 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be427f5-5afe-49f8-8d8a-a1455eee52c2 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:10.677052 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035a843a-59db-4a9e-a12f-2da853d3ea11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:10.677995 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035a843a-59db-4a9e-a12f-2da853d3ea11 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:10.678478 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035a843a-59db-4a9e-a12f-2da853d3ea11 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:10.678966 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035a843a-59db-4a9e-a12f-2da853d3ea11 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:10.679996 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035a843a-59db-4a9e-a12f-2da853d3ea11 None admin] RBAC: Adding query filter params (role.id=af257be2dee54d6b942f58650196582c, scope.project.id=8381c67dd5274d1780bb20df47b105ec, user.id=ca9091e894bc4e01ab24bbbb474a09cd) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:58:10.680499 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035a843a-59db-4a9e-a12f-2da853d3ea11 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:10.682215 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035a843a-59db-4a9e-a12f-2da853d3ea11 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:10.689889 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 49/97] 10.222.0.55 () {56 vars in 1374 bytes} [Fri Sep 23 14:58:10 2022] GET /identity/v3/role_assignments?user.id=ca9091e894bc4e01ab24bbbb474a09cd&scope.project.id=8381c67dd5274d1780bb20df47b105ec&role.id=af257be2dee54d6b942f58650196582c => generated 259 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:58:11.924614 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a0f38c-32ea-4936-8dca-08e66db31b55 None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:11.925133 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a0f38c-32ea-4936-8dca-08e66db31b55 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:11.925209 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a0f38c-32ea-4936-8dca-08e66db31b55 None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:11.926528 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 49/98] 10.222.0.55 () {54 vars in 927 bytes} [Fri Sep 23 14:58:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 14:58:11.937670 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b98b19-542e-4fbc-8c1d-5bf3399f4f06 None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:11.938217 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b98b19-542e-4fbc-8c1d-5bf3399f4f06 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:11.938373 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b98b19-542e-4fbc-8c1d-5bf3399f4f06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:11.960579 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-12b98b19-542e-4fbc-8c1d-5bf3399f4f06 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 14:58:11.961792 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-12b98b19-542e-4fbc-8c1d-5bf3399f4f06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 14:58:12.061965 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 50/99] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Sep 23 14:58:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 14:58:12.267598 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab08894-f48b-44c9-894f-9ad2601ecac0 None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:12.267894 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab08894-f48b-44c9-894f-9ad2601ecac0 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:12.268206 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab08894-f48b-44c9-894f-9ad2601ecac0 None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:12.269558 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 50/100] 10.222.0.55 () {54 vars in 927 bytes} [Fri Sep 23 14:58:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 14:58:12.275623 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3dcc29-d306-46a4-b3f9-4c6c9f615f26 None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:12.276027 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3dcc29-d306-46a4-b3f9-4c6c9f615f26 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:12.276409 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3dcc29-d306-46a4-b3f9-4c6c9f615f26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:12.301916 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-8c3dcc29-d306-46a4-b3f9-4c6c9f615f26 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 14:58:12.303214 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-8c3dcc29-d306-46a4-b3f9-4c6c9f615f26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 14:58:12.333299 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:12.392042 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 51/101] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Sep 23 14:58:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 14:58:12.512065 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab08894-f48b-44c9-894f-9ad2601ecac0 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:12.632679 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab08894-f48b-44c9-894f-9ad2601ecac0 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:12.665402 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:12.671787 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c362314-e641-4a0a-abea-7536f8c7b332 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:12.676727 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c362314-e641-4a0a-abea-7536f8c7b332 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:12.676727 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c362314-e641-4a0a-abea-7536f8c7b332 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:12.676727 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c362314-e641-4a0a-abea-7536f8c7b332 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:12.676727 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c362314-e641-4a0a-abea-7536f8c7b332 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:12.676727 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c362314-e641-4a0a-abea-7536f8c7b332 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:12.677318 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 51/102] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Sep 23 14:58:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:58:12.685256 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3dcc29-d306-46a4-b3f9-4c6c9f615f26 None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:12.733370 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c362314-e641-4a0a-abea-7536f8c7b332 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:12.888904 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3dcc29-d306-46a4-b3f9-4c6c9f615f26 None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:12.933077 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c362314-e641-4a0a-abea-7536f8c7b332 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:12.963759 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a693a7-eb10-44ed-a19a-c612651892f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:12.965086 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a693a7-eb10-44ed-a19a-c612651892f5 None admin] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:12.965370 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a693a7-eb10-44ed-a19a-c612651892f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:12.965649 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a693a7-eb10-44ed-a19a-c612651892f5 None admin] PATH_INFO: `/v3/projects` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:12.968431 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a693a7-eb10-44ed-a19a-c612651892f5 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:12.969921 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a693a7-eb10-44ed-a19a-c612651892f5 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:12.997441 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0edb631-8396-4cc8-8147-7f1ad502cd13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:12.998376 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0edb631-8396-4cc8-8147-7f1ad502cd13 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:12.999007 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0edb631-8396-4cc8-8147-7f1ad502cd13 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:12.999444 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0edb631-8396-4cc8-8147-7f1ad502cd13 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:13.013669 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0edb631-8396-4cc8-8147-7f1ad502cd13 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 14:58:13.020436 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-a0edb631-8396-4cc8-8147-7f1ad502cd13 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 23 14:58:13.020436 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 14:58:13.020436 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 14:58:13.020436 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 14:58:13.020436 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 14:58:13.020436 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 14:58:13.020436 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 14:58:13.020436 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 14:58:13.020436 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 14:58:13.020436 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 14:58:13.020436 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 14:58:13.020436 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 14:58:13.020436 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 14:58:13.020436 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 14:58:13.020436 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 14:58:13.020436 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 14:58:13.021453 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 14:58:13.021453 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 14:58:13.021453 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 14:58:13.021453 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 14:58:13.021453 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 14:58:13.021453 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 14:58:13.021453 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 14:58:13.021453 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 14:58:13.021453 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 14:58:13.021453 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 14:58:13.021453 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 14:58:13.021453 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Sep 23 14:58:13.021453 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 14:58:13.021453 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Sep 23 14:58:13.021453 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 14:58:13.021453 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 14:58:13.022462 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 14:58:13.022462 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Sep 23 14:58:13.022462 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 14:58:13.022462 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Sep 23 14:58:13.022462 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 14:58:13.022462 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 14:58:13.022462 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 14:58:13.022462 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 14:58:13.022462 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 14:58:13.022462 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 14:58:13.022462 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 23 14:58:13.022462 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Sep 23 14:58:13.022462 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 52/103] 10.222.0.55 () {56 vars in 1099 bytes} [Fri Sep 23 14:58:12 2022] GET /identity/v3/roles/reader => generated 84 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 14:58:13.023218 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5018da-bdec-461b-b489-7e1cdd1d9679 None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:13.023665 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5018da-bdec-461b-b489-7e1cdd1d9679 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:13.024361 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5018da-bdec-461b-b489-7e1cdd1d9679 None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:13.027262 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 53/104] 10.222.0.55 () {54 vars in 927 bytes} [Fri Sep 23 14:58:13 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 14:58:13.028910 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251a3443-9f38-4f6b-87da-f75af1f6edd8 None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:13.029550 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251a3443-9f38-4f6b-87da-f75af1f6edd8 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:13.029832 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251a3443-9f38-4f6b-87da-f75af1f6edd8 None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:13.031446 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 54/105] 10.222.0.55 () {54 vars in 927 bytes} [Fri Sep 23 14:58:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 14:58:13.032396 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251a3443-9f38-4f6b-87da-f75af1f6edd8 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:13.075156 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 52/106] 10.222.0.55 () {60 vars in 1141 bytes} [Fri Sep 23 14:58:12 2022] POST /identity/v3/projects => generated 302 bytes in 390 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 14:58:13.085943 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab825929-8a04-4848-be35-d0a15d86edbb None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:13.086663 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab825929-8a04-4848-be35-d0a15d86edbb None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:13.090317 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab825929-8a04-4848-be35-d0a15d86edbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:13.124786 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-ab825929-8a04-4848-be35-d0a15d86edbb None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 14:58:13.126435 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-ab825929-8a04-4848-be35-d0a15d86edbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 14:58:13.135614 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:13.148091 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:13.220209 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251a3443-9f38-4f6b-87da-f75af1f6edd8 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:13.276594 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 53/107] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Sep 23 14:58:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 14:58:13.278306 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1691957c-c0cc-49b8-b4c0-b0bd1bd85f4f None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:13.278802 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1691957c-c0cc-49b8-b4c0-b0bd1bd85f4f None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:13.279276 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1691957c-c0cc-49b8-b4c0-b0bd1bd85f4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:13.297057 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019d25f3-25e0-47d9-a542-1c1dcf880ce5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:13.298498 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019d25f3-25e0-47d9-a542-1c1dcf880ce5 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:13.298952 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019d25f3-25e0-47d9-a542-1c1dcf880ce5 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:13.300639 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019d25f3-25e0-47d9-a542-1c1dcf880ce5 None admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:13.313533 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019d25f3-25e0-47d9-a542-1c1dcf880ce5 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:58:13.313533 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019d25f3-25e0-47d9-a542-1c1dcf880ce5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:13.313533 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019d25f3-25e0-47d9-a542-1c1dcf880ce5 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:13.313785 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-1691957c-c0cc-49b8-b4c0-b0bd1bd85f4f None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 14:58:13.313785 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-1691957c-c0cc-49b8-b4c0-b0bd1bd85f4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 14:58:13.326494 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 55/108] 10.222.0.55 () {56 vars in 1101 bytes} [Fri Sep 23 14:58:13 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:58:13.337076 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019d25f3-25e0-47d9-a542-1c1dcf880ce5 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:13.423508 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 54/109] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Sep 23 14:58:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 14:58:13.429069 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76838b62-3143-46ea-91dd-7834d7d07524 None None] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:13.429465 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76838b62-3143-46ea-91dd-7834d7d07524 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:13.429897 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76838b62-3143-46ea-91dd-7834d7d07524 None None] PATH_INFO: `/` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:13.431894 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 55/110] 10.222.0.55 () {54 vars in 927 bytes} [Fri Sep 23 14:58:13 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 14:58:13.437664 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6a1521-e23f-4382-bad1-a211da62af58 None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:13.438076 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6a1521-e23f-4382-bad1-a211da62af58 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:13.438476 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6a1521-e23f-4382-bad1-a211da62af58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:13.464329 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019d25f3-25e0-47d9-a542-1c1dcf880ce5 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:13.480838 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-9e6a1521-e23f-4382-bad1-a211da62af58 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 14:58:13.480838 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-9e6a1521-e23f-4382-bad1-a211da62af58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 14:58:13.490885 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:13.506686 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb945fa-967d-4ce9-8e76-00c85afb1f69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:13.507668 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb945fa-967d-4ce9-8e76-00c85afb1f69 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:13.508423 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb945fa-967d-4ce9-8e76-00c85afb1f69 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:13.509376 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb945fa-967d-4ce9-8e76-00c85afb1f69 None admin] PATH_INFO: `/v3/users/fc9052ae566f4bd2ab726f74f7a156cd` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:13.529980 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb945fa-967d-4ce9-8e76-00c85afb1f69 None admin] RBAC: Authorizing `identity:get_user(user_id=fc9052ae566f4bd2ab726f74f7a156cd)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:13.532329 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb945fa-967d-4ce9-8e76-00c85afb1f69 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:13.534872 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 56/111] 10.222.0.55 () {56 vars in 1177 bytes} [Fri Sep 23 14:58:13 2022] GET /identity/v3/users/fc9052ae566f4bd2ab726f74f7a156cd => generated 289 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:58:13.542672 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb945fa-967d-4ce9-8e76-00c85afb1f69 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:13.617660 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 56/112] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Sep 23 14:58:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 14:58:13.655802 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb945fa-967d-4ce9-8e76-00c85afb1f69 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:13.659894 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6a1521-e23f-4382-bad1-a211da62af58 None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:13.662696 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:13.694809 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe03924-2294-4abd-9671-0863382bcbaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:13.695688 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe03924-2294-4abd-9671-0863382bcbaa None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:13.696154 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe03924-2294-4abd-9671-0863382bcbaa None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:13.696604 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe03924-2294-4abd-9671-0863382bcbaa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:13.771779 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:13.774059 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe03924-2294-4abd-9671-0863382bcbaa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:13.777229 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe03924-2294-4abd-9671-0863382bcbaa None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:13.930682 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6a1521-e23f-4382-bad1-a211da62af58 None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:13.969949 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 57/113] 10.222.0.55 () {58 vars in 1282 bytes} [Fri Sep 23 14:58:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 23 14:58:13.970885 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe03924-2294-4abd-9671-0863382bcbaa None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:13.978732 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f31548-3f1d-4c8d-923a-31dd4a8df8b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:13.979385 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f31548-3f1d-4c8d-923a-31dd4a8df8b9 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:13.979672 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f31548-3f1d-4c8d-923a-31dd4a8df8b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:13.980579 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f31548-3f1d-4c8d-923a-31dd4a8df8b9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:13.995343 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f31548-3f1d-4c8d-923a-31dd4a8df8b9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 14:58:14.003647 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application [None req-98f31548-3f1d-4c8d-923a-31dd4a8df8b9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 23 14:58:14.003647 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 14:58:14.003647 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 14:58:14.003647 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 14:58:14.003647 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 14:58:14.003647 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 14:58:14.003647 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 14:58:14.003647 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 14:58:14.003647 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 14:58:14.003647 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 14:58:14.003647 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 14:58:14.003647 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 14:58:14.003647 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 14:58:14.003647 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 14:58:14.003647 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 14:58:14.003647 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 14:58:14.004656 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 14:58:14.004656 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 14:58:14.004656 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 14:58:14.004656 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 14:58:14.004656 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 14:58:14.004656 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 14:58:14.004656 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 14:58:14.004656 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 14:58:14.004656 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 14:58:14.004656 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 14:58:14.004656 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 14:58:14.004656 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application with Lock( Sep 23 14:58:14.004656 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 14:58:14.004656 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._enter() Sep 23 14:58:14.004656 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 14:58:14.004656 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 14:58:14.005595 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 14:58:14.005595 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.creator() Sep 23 14:58:14.005595 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 14:58:14.005595 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application created_value = creator( Sep 23 14:58:14.005595 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 14:58:14.005595 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 14:58:14.005595 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 14:58:14.005595 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 14:58:14.005595 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 14:58:14.005595 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 14:58:14.005595 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 23 14:58:14.005595 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Sep 23 14:58:14.006193 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 57/114] 10.222.0.55 () {56 vars in 1099 bytes} [Fri Sep 23 14:58:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 347 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 14:58:14.011703 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f31548-3f1d-4c8d-923a-31dd4a8df8b9 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:14.072638 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:14.097339 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:14.116616 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe03924-2294-4abd-9671-0863382bcbaa None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:14.142165 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f31548-3f1d-4c8d-923a-31dd4a8df8b9 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:14.168212 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0ae062-bfa2-4b41-9dec-76c41548f7ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:14.169779 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0ae062-bfa2-4b41-9dec-76c41548f7ce None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:14.170533 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0ae062-bfa2-4b41-9dec-76c41548f7ce None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:14.171044 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0ae062-bfa2-4b41-9dec-76c41548f7ce None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:14.182620 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcf4bf1-3c82-408d-8cca-f3996b8d1048 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:14.183771 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcf4bf1-3c82-408d-8cca-f3996b8d1048 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:14.184152 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcf4bf1-3c82-408d-8cca-f3996b8d1048 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:14.184500 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcf4bf1-3c82-408d-8cca-f3996b8d1048 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:14.187494 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0ae062-bfa2-4b41-9dec-76c41548f7ce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 14:58:14.193995 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-9d0ae062-bfa2-4b41-9dec-76c41548f7ce None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 23 14:58:14.193995 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 14:58:14.193995 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 14:58:14.193995 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 14:58:14.193995 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 14:58:14.193995 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 14:58:14.193995 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 14:58:14.193995 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 14:58:14.193995 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 14:58:14.193995 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 14:58:14.193995 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 14:58:14.193995 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 14:58:14.193995 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 14:58:14.193995 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 14:58:14.193995 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 14:58:14.193995 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 14:58:14.194909 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 14:58:14.194909 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 14:58:14.194909 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 14:58:14.194909 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 14:58:14.194909 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 14:58:14.194909 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 14:58:14.194909 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 14:58:14.194909 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 14:58:14.194909 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 14:58:14.194909 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 14:58:14.194909 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 14:58:14.194909 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Sep 23 14:58:14.194909 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 14:58:14.194909 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Sep 23 14:58:14.194909 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 14:58:14.194909 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 14:58:14.195690 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 14:58:14.195690 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Sep 23 14:58:14.195690 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 14:58:14.195690 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Sep 23 14:58:14.195690 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 14:58:14.195690 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 14:58:14.195690 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 14:58:14.195690 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 14:58:14.195690 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 14:58:14.195690 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 14:58:14.195690 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 23 14:58:14.195690 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Sep 23 14:58:14.198251 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 58/115] 10.222.0.55 () {56 vars in 1096 bytes} [Fri Sep 23 14:58:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 14:58:14.198894 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0ae062-bfa2-4b41-9dec-76c41548f7ce None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:14.202296 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcf4bf1-3c82-408d-8cca-f3996b8d1048 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 14:58:14.208795 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application [None req-0bcf4bf1-3c82-408d-8cca-f3996b8d1048 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 23 14:58:14.208795 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 14:58:14.208795 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 14:58:14.208795 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 14:58:14.208795 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 14:58:14.208795 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 14:58:14.208795 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 14:58:14.208795 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 14:58:14.208795 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 14:58:14.208795 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 14:58:14.208795 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 14:58:14.208795 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 14:58:14.208795 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 14:58:14.208795 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 14:58:14.208795 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 14:58:14.208795 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 14:58:14.209706 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 14:58:14.209706 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 14:58:14.209706 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 14:58:14.209706 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 14:58:14.209706 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 14:58:14.209706 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 14:58:14.209706 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 14:58:14.209706 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 14:58:14.209706 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 14:58:14.209706 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 14:58:14.209706 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 14:58:14.209706 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application with Lock( Sep 23 14:58:14.209706 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 14:58:14.209706 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._enter() Sep 23 14:58:14.209706 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 14:58:14.209706 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 14:58:14.210886 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 14:58:14.210886 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.creator() Sep 23 14:58:14.210886 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 14:58:14.210886 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application created_value = creator( Sep 23 14:58:14.210886 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 14:58:14.210886 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 14:58:14.210886 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 14:58:14.210886 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 14:58:14.210886 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 14:58:14.210886 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 14:58:14.210886 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 23 14:58:14.210886 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Sep 23 14:58:14.216291 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 58/116] 10.222.0.55 () {56 vars in 1114 bytes} [Fri Sep 23 14:58:14 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 14:58:14.217260 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcf4bf1-3c82-408d-8cca-f3996b8d1048 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:14.333270 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0ae062-bfa2-4b41-9dec-76c41548f7ce None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:14.345687 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcf4bf1-3c82-408d-8cca-f3996b8d1048 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:14.371017 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07faa86-8909-4c83-acb0-7e6df687d985 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:14.371351 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:14.371656 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07faa86-8909-4c83-acb0-7e6df687d985 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:14.371962 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07faa86-8909-4c83-acb0-7e6df687d985 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:14.372273 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07faa86-8909-4c83-acb0-7e6df687d985 None admin] PATH_INFO: `/v3/projects/0ffbe3ca86954853be08ecf627d6af00` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:14.373711 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07faa86-8909-4c83-acb0-7e6df687d985 None admin] RBAC: Authorizing `identity:get_project(project_id=0ffbe3ca86954853be08ecf627d6af00)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:14.375166 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07faa86-8909-4c83-acb0-7e6df687d985 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:14.376826 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 59/117] 10.222.0.55 () {56 vars in 1186 bytes} [Fri Sep 23 14:58:14 2022] GET /identity/v3/projects/0ffbe3ca86954853be08ecf627d6af00 => generated 298 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:58:14.377664 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07faa86-8909-4c83-acb0-7e6df687d985 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:14.387587 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3cd1b9-20b0-464f-9be0-fb971af583ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:14.388503 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3cd1b9-20b0-464f-9be0-fb971af583ea None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:14.388648 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:14.388797 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3cd1b9-20b0-464f-9be0-fb971af583ea None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:14.389128 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3cd1b9-20b0-464f-9be0-fb971af583ea None admin] PATH_INFO: `/v3/roles` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:14.392144 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3cd1b9-20b0-464f-9be0-fb971af583ea None admin] RBAC: Adding query filter params (name=member) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:58:14.392144 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3cd1b9-20b0-464f-9be0-fb971af583ea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:14.392144 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3cd1b9-20b0-464f-9be0-fb971af583ea None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:14.406352 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 59/118] 10.222.0.55 () {56 vars in 1101 bytes} [Fri Sep 23 14:58:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:58:14.406352 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3cd1b9-20b0-464f-9be0-fb971af583ea None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:14.491729 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07faa86-8909-4c83-acb0-7e6df687d985 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:14.513730 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3cd1b9-20b0-464f-9be0-fb971af583ea None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:14.529420 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8ccae9-1eda-4946-8522-bc92780cbd94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:14.530035 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8ccae9-1eda-4946-8522-bc92780cbd94 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:14.530321 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8ccae9-1eda-4946-8522-bc92780cbd94 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:14.530599 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8ccae9-1eda-4946-8522-bc92780cbd94 None admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:14.531270 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8ccae9-1eda-4946-8522-bc92780cbd94 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:58:14.531592 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8ccae9-1eda-4946-8522-bc92780cbd94 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:14.533263 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8ccae9-1eda-4946-8522-bc92780cbd94 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:14.543275 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 60/119] 10.222.0.55 () {56 vars in 1099 bytes} [Fri Sep 23 14:58:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:58:14.549095 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dc7482-6b9d-4e15-b9fa-4042669d887a None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:14.549394 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dc7482-6b9d-4e15-b9fa-4042669d887a None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:14.549673 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dc7482-6b9d-4e15-b9fa-4042669d887a None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:14.550916 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 61/120] 10.222.0.55 () {54 vars in 927 bytes} [Fri Sep 23 14:58:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 14:58:14.554510 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12991d93-a09b-4fe3-8a5b-4a8c8f83e357 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:14.555441 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12991d93-a09b-4fe3-8a5b-4a8c8f83e357 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:14.555596 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7dc7482-6b9d-4e15-b9fa-4042669d887a None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:14.556206 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12991d93-a09b-4fe3-8a5b-4a8c8f83e357 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:14.556674 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12991d93-a09b-4fe3-8a5b-4a8c8f83e357 None admin] PATH_INFO: `/v3/roles` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:14.557519 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12991d93-a09b-4fe3-8a5b-4a8c8f83e357 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:58:14.558013 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12991d93-a09b-4fe3-8a5b-4a8c8f83e357 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:14.560632 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12991d93-a09b-4fe3-8a5b-4a8c8f83e357 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:14.569664 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 60/121] 10.222.0.55 () {56 vars in 1111 bytes} [Fri Sep 23 14:58:14 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:58:14.569664 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12991d93-a09b-4fe3-8a5b-4a8c8f83e357 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:14.622984 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:14.632638 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:14.669064 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7dc7482-6b9d-4e15-b9fa-4042669d887a None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:14.680741 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12991d93-a09b-4fe3-8a5b-4a8c8f83e357 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:14.709159 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cee192-2409-4006-b071-6963276893c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:14.709781 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cee192-2409-4006-b071-6963276893c9 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:14.710319 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cee192-2409-4006-b071-6963276893c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:14.710603 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cee192-2409-4006-b071-6963276893c9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:14.712687 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63cee192-2409-4006-b071-6963276893c9 None admin] RBAC: Adding query filter params (role.id=ea002840090d4194bb75fa083a55dc5f, scope.project.id=0ffbe3ca86954853be08ecf627d6af00, user.id=fc9052ae566f4bd2ab726f74f7a156cd) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:58:14.713510 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63cee192-2409-4006-b071-6963276893c9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:14.714985 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63cee192-2409-4006-b071-6963276893c9 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:14.716295 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6bb459-18c2-4e8c-b591-f37bdf1a6e9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:14.717000 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6bb459-18c2-4e8c-b591-f37bdf1a6e9f None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:14.717352 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6bb459-18c2-4e8c-b591-f37bdf1a6e9f None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:14.717698 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6bb459-18c2-4e8c-b591-f37bdf1a6e9f None admin] PATH_INFO: `/v3/users/ca9091e894bc4e01ab24bbbb474a09cd` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:14.719030 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6bb459-18c2-4e8c-b591-f37bdf1a6e9f None admin] RBAC: Authorizing `identity:get_user(user_id=ca9091e894bc4e01ab24bbbb474a09cd)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:14.720600 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6bb459-18c2-4e8c-b591-f37bdf1a6e9f None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:14.732629 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 62/122] 10.222.0.55 () {56 vars in 1374 bytes} [Fri Sep 23 14:58:14 2022] GET /identity/v3/role_assignments?user.id=fc9052ae566f4bd2ab726f74f7a156cd&scope.project.id=0ffbe3ca86954853be08ecf627d6af00&role.id=ea002840090d4194bb75fa083a55dc5f => generated 259 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:58:14.732629 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cee192-2409-4006-b071-6963276893c9 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:14.732892 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 61/123] 10.222.0.55 () {56 vars in 1177 bytes} [Fri Sep 23 14:58:14 2022] GET /identity/v3/users/ca9091e894bc4e01ab24bbbb474a09cd => generated 275 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:58:14.732892 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c10b02-4aa3-498e-b692-c38ed9a0db72 None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:14.732892 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c10b02-4aa3-498e-b692-c38ed9a0db72 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:14.732892 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c10b02-4aa3-498e-b692-c38ed9a0db72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:14.749402 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-20c10b02-4aa3-498e-b692-c38ed9a0db72 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 14:58:14.750806 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-20c10b02-4aa3-498e-b692-c38ed9a0db72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 14:58:14.851320 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 62/124] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Sep 23 14:58:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 14:58:14.853505 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c10b02-4aa3-498e-b692-c38ed9a0db72 None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:14.875180 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cee192-2409-4006-b071-6963276893c9 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:14.942253 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:14.943446 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:14.951159 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3297f14d-d56c-446d-8e5e-39391e49c66c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:14.951401 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3297f14d-d56c-446d-8e5e-39391e49c66c None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:14.959229 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3297f14d-d56c-446d-8e5e-39391e49c66c None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:14.959229 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3297f14d-d56c-446d-8e5e-39391e49c66c None admin] PATH_INFO: `/v3/users/0fc10e0211a54a0faf7ea9b86542de03` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:14.959229 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3297f14d-d56c-446d-8e5e-39391e49c66c None admin] RBAC: Authorizing `identity:get_user(user_id=0fc10e0211a54a0faf7ea9b86542de03)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:14.959229 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3297f14d-d56c-446d-8e5e-39391e49c66c None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:14.959229 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 63/125] 10.222.0.55 () {56 vars in 1177 bytes} [Fri Sep 23 14:58:14 2022] GET /identity/v3/users/0fc10e0211a54a0faf7ea9b86542de03 => generated 247 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:58:14.959229 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3297f14d-d56c-446d-8e5e-39391e49c66c None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:15.036651 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c10b02-4aa3-498e-b692-c38ed9a0db72 None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:15.074957 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39538fd4-fbb2-4c63-b4da-22cd1a900c82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:15.075652 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39538fd4-fbb2-4c63-b4da-22cd1a900c82 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:15.075941 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39538fd4-fbb2-4c63-b4da-22cd1a900c82 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:15.076248 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39538fd4-fbb2-4c63-b4da-22cd1a900c82 None admin] PATH_INFO: `/v3/users/ca9091e894bc4e01ab24bbbb474a09cd` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:15.079264 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39538fd4-fbb2-4c63-b4da-22cd1a900c82 None admin] RBAC: Authorizing `identity:get_user(user_id=ca9091e894bc4e01ab24bbbb474a09cd)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:15.079264 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39538fd4-fbb2-4c63-b4da-22cd1a900c82 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:15.081633 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 63/126] 10.222.0.55 () {56 vars in 1177 bytes} [Fri Sep 23 14:58:14 2022] GET /identity/v3/users/ca9091e894bc4e01ab24bbbb474a09cd => generated 275 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:58:15.081633 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39538fd4-fbb2-4c63-b4da-22cd1a900c82 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:15.187106 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3297f14d-d56c-446d-8e5e-39391e49c66c None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:15.190877 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39538fd4-fbb2-4c63-b4da-22cd1a900c82 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:15.205619 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:15.227831 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398672db-abfb-4827-917d-be684eaf2765 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:15.228588 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398672db-abfb-4827-917d-be684eaf2765 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:15.228942 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398672db-abfb-4827-917d-be684eaf2765 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:15.229290 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398672db-abfb-4827-917d-be684eaf2765 None admin] PATH_INFO: `/v3/projects/0ffbe3ca86954853be08ecf627d6af00` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:15.230668 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398672db-abfb-4827-917d-be684eaf2765 None admin] RBAC: Authorizing `identity:get_project(project_id=0ffbe3ca86954853be08ecf627d6af00)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:15.232221 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398672db-abfb-4827-917d-be684eaf2765 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:15.234264 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 64/127] 10.222.0.55 () {56 vars in 1186 bytes} [Fri Sep 23 14:58:15 2022] GET /identity/v3/projects/0ffbe3ca86954853be08ecf627d6af00 => generated 298 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:58:15.234988 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398672db-abfb-4827-917d-be684eaf2765 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:15.277384 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c699601-80bd-4d46-af94-da64f03c25d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:15.278443 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c699601-80bd-4d46-af94-da64f03c25d2 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:15.278926 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c699601-80bd-4d46-af94-da64f03c25d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:15.279398 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c699601-80bd-4d46-af94-da64f03c25d2 None admin] PATH_INFO: `/v3/projects/0ffbe3ca86954853be08ecf627d6af00` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:15.280844 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c699601-80bd-4d46-af94-da64f03c25d2 None admin] RBAC: Authorizing `identity:get_project(project_id=0ffbe3ca86954853be08ecf627d6af00)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:15.282873 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c699601-80bd-4d46-af94-da64f03c25d2 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:15.285106 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 64/128] 10.222.0.55 () {56 vars in 1186 bytes} [Fri Sep 23 14:58:14 2022] GET /identity/v3/projects/0ffbe3ca86954853be08ecf627d6af00 => generated 298 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:58:15.286373 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c699601-80bd-4d46-af94-da64f03c25d2 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:15.350758 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398672db-abfb-4827-917d-be684eaf2765 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:15.360910 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:15.390800 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf0e219-8150-455f-9f34-f2dca10cfbb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:15.392006 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf0e219-8150-455f-9f34-f2dca10cfbb8 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:15.392276 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf0e219-8150-455f-9f34-f2dca10cfbb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:15.392555 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf0e219-8150-455f-9f34-f2dca10cfbb8 None admin] PATH_INFO: `/v3/projects/0ffbe3ca86954853be08ecf627d6af00` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:15.393743 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf0e219-8150-455f-9f34-f2dca10cfbb8 None admin] RBAC: Authorizing `identity:get_project(project_id=0ffbe3ca86954853be08ecf627d6af00)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:15.395341 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf0e219-8150-455f-9f34-f2dca10cfbb8 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:15.397121 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 65/129] 10.222.0.55 () {56 vars in 1186 bytes} [Fri Sep 23 14:58:15 2022] GET /identity/v3/projects/0ffbe3ca86954853be08ecf627d6af00 => generated 298 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:58:15.398327 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf0e219-8150-455f-9f34-f2dca10cfbb8 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:15.409886 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c699601-80bd-4d46-af94-da64f03c25d2 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:15.449356 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a07306f-ed65-4149-bb9f-dc463b23ebde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:15.450443 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a07306f-ed65-4149-bb9f-dc463b23ebde None admin] REQUEST_METHOD: `PUT` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:15.450925 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a07306f-ed65-4149-bb9f-dc463b23ebde None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:15.451397 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a07306f-ed65-4149-bb9f-dc463b23ebde None admin] PATH_INFO: `/v3/projects/0ffbe3ca86954853be08ecf627d6af00/users/0fc10e0211a54a0faf7ea9b86542de03/roles/8e2919e79df040e8926f21607818b58c` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:15.452898 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:15.454914 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a07306f-ed65-4149-bb9f-dc463b23ebde None admin] RBAC: Authorizing `identity:create_grant(project_id=0ffbe3ca86954853be08ecf627d6af00, user_id=0fc10e0211a54a0faf7ea9b86542de03, role_id=8e2919e79df040e8926f21607818b58c)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:15.456780 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a07306f-ed65-4149-bb9f-dc463b23ebde None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:15.512856 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf0e219-8150-455f-9f34-f2dca10cfbb8 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:15.534498 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 65/130] 10.222.0.55 () {58 vars in 1439 bytes} [Fri Sep 23 14:58:15 2022] PUT /identity/v3/projects/0ffbe3ca86954853be08ecf627d6af00/users/0fc10e0211a54a0faf7ea9b86542de03/roles/8e2919e79df040e8926f21607818b58c => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 14:58:15.536444 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a07306f-ed65-4149-bb9f-dc463b23ebde None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:15.578061 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe4fdf8-5e9c-4607-b52f-5c2c8e41c4e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:15.579223 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe4fdf8-5e9c-4607-b52f-5c2c8e41c4e8 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:15.580186 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe4fdf8-5e9c-4607-b52f-5c2c8e41c4e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:15.581277 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe4fdf8-5e9c-4607-b52f-5c2c8e41c4e8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:15.611640 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe4fdf8-5e9c-4607-b52f-5c2c8e41c4e8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 14:58:15.620013 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application [None req-5fe4fdf8-5e9c-4607-b52f-5c2c8e41c4e8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 23 14:58:15.620013 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 14:58:15.620013 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 14:58:15.620013 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 14:58:15.620013 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 14:58:15.620013 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 14:58:15.620013 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 14:58:15.620013 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 14:58:15.620013 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 14:58:15.620013 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 14:58:15.620013 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 14:58:15.620013 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 14:58:15.620013 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 14:58:15.620013 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 14:58:15.620013 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 14:58:15.620013 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 14:58:15.620986 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 14:58:15.620986 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 14:58:15.620986 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 14:58:15.620986 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 14:58:15.620986 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 14:58:15.620986 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 14:58:15.620986 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 14:58:15.620986 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 14:58:15.620986 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 14:58:15.620986 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 14:58:15.620986 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 14:58:15.620986 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application with Lock( Sep 23 14:58:15.620986 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 14:58:15.620986 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._enter() Sep 23 14:58:15.620986 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 14:58:15.620986 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 14:58:15.622252 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 14:58:15.622252 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.creator() Sep 23 14:58:15.622252 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 14:58:15.622252 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application created_value = creator( Sep 23 14:58:15.622252 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 14:58:15.622252 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 14:58:15.622252 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 14:58:15.622252 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 14:58:15.622252 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 14:58:15.622252 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 14:58:15.622252 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 23 14:58:15.622252 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Sep 23 14:58:15.636451 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 66/131] 10.222.0.55 () {56 vars in 1099 bytes} [Fri Sep 23 14:58:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 14:58:15.637846 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe4fdf8-5e9c-4607-b52f-5c2c8e41c4e8 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:15.769792 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a07306f-ed65-4149-bb9f-dc463b23ebde None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:15.780735 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe4fdf8-5e9c-4607-b52f-5c2c8e41c4e8 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:15.795083 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:15.803018 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:15.814658 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57785d0-1125-4346-9705-086b1d75363b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:15.815369 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57785d0-1125-4346-9705-086b1d75363b None admin] REQUEST_METHOD: `PUT` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:15.820022 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57785d0-1125-4346-9705-086b1d75363b None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:15.820022 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57785d0-1125-4346-9705-086b1d75363b None admin] PATH_INFO: `/v3/projects/0ffbe3ca86954853be08ecf627d6af00/users/ca9091e894bc4e01ab24bbbb474a09cd/roles/af257be2dee54d6b942f58650196582c` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:15.820022 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57785d0-1125-4346-9705-086b1d75363b None admin] RBAC: Authorizing `identity:create_grant(project_id=0ffbe3ca86954853be08ecf627d6af00, user_id=ca9091e894bc4e01ab24bbbb474a09cd, role_id=af257be2dee54d6b942f58650196582c)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:15.824873 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57785d0-1125-4346-9705-086b1d75363b None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:15.831127 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26eec5f0-3cd6-47d3-aaa6-3c74c94b591e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:15.831854 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26eec5f0-3cd6-47d3-aaa6-3c74c94b591e None admin] REQUEST_METHOD: `PUT` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:15.832174 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26eec5f0-3cd6-47d3-aaa6-3c74c94b591e None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:15.837712 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26eec5f0-3cd6-47d3-aaa6-3c74c94b591e None admin] PATH_INFO: `/v3/projects/0ffbe3ca86954853be08ecf627d6af00/users/ca9091e894bc4e01ab24bbbb474a09cd/roles/5c74fdc0e9cc44538cd8282006ed4443` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:15.837712 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26eec5f0-3cd6-47d3-aaa6-3c74c94b591e None admin] RBAC: Authorizing `identity:create_grant(project_id=0ffbe3ca86954853be08ecf627d6af00, user_id=ca9091e894bc4e01ab24bbbb474a09cd, role_id=5c74fdc0e9cc44538cd8282006ed4443)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:15.837712 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26eec5f0-3cd6-47d3-aaa6-3c74c94b591e None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:15.977340 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 66/132] 10.222.0.55 () {58 vars in 1439 bytes} [Fri Sep 23 14:58:15 2022] PUT /identity/v3/projects/0ffbe3ca86954853be08ecf627d6af00/users/ca9091e894bc4e01ab24bbbb474a09cd/roles/af257be2dee54d6b942f58650196582c => generated 0 bytes in 442 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 14:58:15.978559 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57785d0-1125-4346-9705-086b1d75363b None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:16.018397 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 67/133] 10.222.0.55 () {58 vars in 1439 bytes} [Fri Sep 23 14:58:15 2022] PUT /identity/v3/projects/0ffbe3ca86954853be08ecf627d6af00/users/ca9091e894bc4e01ab24bbbb474a09cd/roles/5c74fdc0e9cc44538cd8282006ed4443 => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 14:58:16.028491 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8c005e-6ed0-47e9-9113-108ae7d17e64 None None] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:16.028491 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8c005e-6ed0-47e9-9113-108ae7d17e64 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:16.028491 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8c005e-6ed0-47e9-9113-108ae7d17e64 None None] PATH_INFO: `/` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:16.029207 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 68/134] 10.222.0.55 () {54 vars in 927 bytes} [Fri Sep 23 14:58:16 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 14:58:16.035626 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a51dfff-6161-455f-8018-81ec5d65b261 None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:16.036079 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a51dfff-6161-455f-8018-81ec5d65b261 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:16.036233 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a51dfff-6161-455f-8018-81ec5d65b261 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:16.101420 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-6a51dfff-6161-455f-8018-81ec5d65b261 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 14:58:16.103190 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-6a51dfff-6161-455f-8018-81ec5d65b261 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 14:58:16.189299 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57785d0-1125-4346-9705-086b1d75363b None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:16.235634 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef30535d-8596-442b-b319-19b28e274fd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:16.236281 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef30535d-8596-442b-b319-19b28e274fd9 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:16.236586 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef30535d-8596-442b-b319-19b28e274fd9 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:16.236931 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef30535d-8596-442b-b319-19b28e274fd9 None admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:16.238479 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef30535d-8596-442b-b319-19b28e274fd9 None admin] RBAC: Adding query filter params (name=member) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:58:16.238896 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef30535d-8596-442b-b319-19b28e274fd9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:16.240698 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef30535d-8596-442b-b319-19b28e274fd9 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:16.251605 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 67/135] 10.222.0.55 () {56 vars in 1101 bytes} [Fri Sep 23 14:58:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:58:16.257663 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef30535d-8596-442b-b319-19b28e274fd9 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:16.258176 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 69/136] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Sep 23 14:58:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 23 14:58:16.335898 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:16.403925 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef30535d-8596-442b-b319-19b28e274fd9 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:16.444689 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b20c56a-e676-408b-b541-a93c5a202447 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:16.445330 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b20c56a-e676-408b-b541-a93c5a202447 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:16.445612 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b20c56a-e676-408b-b541-a93c5a202447 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:16.445886 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b20c56a-e676-408b-b541-a93c5a202447 None admin] PATH_INFO: `/v3/users/ca9091e894bc4e01ab24bbbb474a09cd` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:16.447568 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b20c56a-e676-408b-b541-a93c5a202447 None admin] RBAC: Authorizing `identity:get_user(user_id=ca9091e894bc4e01ab24bbbb474a09cd)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:16.449063 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b20c56a-e676-408b-b541-a93c5a202447 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:16.450696 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 68/137] 10.222.0.55 () {56 vars in 1177 bytes} [Fri Sep 23 14:58:16 2022] GET /identity/v3/users/ca9091e894bc4e01ab24bbbb474a09cd => generated 275 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 14:58:16.456438 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a51dfff-6161-455f-8018-81ec5d65b261 None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:16.565968 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:16.570475 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a51dfff-6161-455f-8018-81ec5d65b261 None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:16.611352 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac4ff08-c891-4577-b6f8-cd525fc5a875 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:16.612052 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac4ff08-c891-4577-b6f8-cd525fc5a875 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:16.617565 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac4ff08-c891-4577-b6f8-cd525fc5a875 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:16.617565 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac4ff08-c891-4577-b6f8-cd525fc5a875 None admin] PATH_INFO: `/v3/projects/8381c67dd5274d1780bb20df47b105ec` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:16.617565 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac4ff08-c891-4577-b6f8-cd525fc5a875 None admin] RBAC: Authorizing `identity:get_project(project_id=8381c67dd5274d1780bb20df47b105ec)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:16.617565 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac4ff08-c891-4577-b6f8-cd525fc5a875 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:16.621765 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 70/138] 10.222.0.55 () {56 vars in 1186 bytes} [Fri Sep 23 14:58:16 2022] GET /identity/v3/projects/8381c67dd5274d1780bb20df47b105ec => generated 312 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:58:16.623121 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b20c56a-e676-408b-b541-a93c5a202447 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:16.718506 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac4ff08-c891-4577-b6f8-cd525fc5a875 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:16.737131 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b20c56a-e676-408b-b541-a93c5a202447 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:16.768359 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:16.779735 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149aa3d8-aa67-4da5-a206-9fec9e43a915 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:16.790739 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149aa3d8-aa67-4da5-a206-9fec9e43a915 None admin] REQUEST_METHOD: `PUT` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:16.790739 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149aa3d8-aa67-4da5-a206-9fec9e43a915 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:16.790739 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149aa3d8-aa67-4da5-a206-9fec9e43a915 None admin] PATH_INFO: `/v3/projects/8381c67dd5274d1780bb20df47b105ec/users/ca9091e894bc4e01ab24bbbb474a09cd/roles/af257be2dee54d6b942f58650196582c` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:16.790739 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149aa3d8-aa67-4da5-a206-9fec9e43a915 None admin] RBAC: Authorizing `identity:create_grant(project_id=8381c67dd5274d1780bb20df47b105ec, user_id=ca9091e894bc4e01ab24bbbb474a09cd, role_id=af257be2dee54d6b942f58650196582c)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:16.790739 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149aa3d8-aa67-4da5-a206-9fec9e43a915 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:16.876896 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 69/139] 10.222.0.55 () {58 vars in 1439 bytes} [Fri Sep 23 14:58:16 2022] PUT /identity/v3/projects/8381c67dd5274d1780bb20df47b105ec/users/ca9091e894bc4e01ab24bbbb474a09cd/roles/af257be2dee54d6b942f58650196582c => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 14:58:16.924254 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac4ff08-c891-4577-b6f8-cd525fc5a875 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:16.980980 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56fdffc-570e-444c-902a-205366be1877 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:16.981938 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56fdffc-570e-444c-902a-205366be1877 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:16.982402 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56fdffc-570e-444c-902a-205366be1877 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:16.982868 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56fdffc-570e-444c-902a-205366be1877 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:16.984220 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56fdffc-570e-444c-902a-205366be1877 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:16.985865 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56fdffc-570e-444c-902a-205366be1877 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:16.987664 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 71/140] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Sep 23 14:58:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:58:16.993920 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149aa3d8-aa67-4da5-a206-9fec9e43a915 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:17.158865 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149aa3d8-aa67-4da5-a206-9fec9e43a915 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:17.202673 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71c045f-1d0b-4fff-9114-ab027d607083 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:17.203324 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71c045f-1d0b-4fff-9114-ab027d607083 None admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:17.203585 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71c045f-1d0b-4fff-9114-ab027d607083 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:17.203868 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71c045f-1d0b-4fff-9114-ab027d607083 None admin] PATH_INFO: `/v3/users` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:17.204664 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71c045f-1d0b-4fff-9114-ab027d607083 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:17.206071 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71c045f-1d0b-4fff-9114-ab027d607083 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:17.320270 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 70/141] 10.222.0.55 () {60 vars in 1132 bytes} [Fri Sep 23 14:58:16 2022] POST /identity/v3/users => generated 283 bytes in 327 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 23 14:58:17.527534 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c7e986-bbdb-4f0b-9772-dff7cb607015 None None] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:17.527534 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c7e986-bbdb-4f0b-9772-dff7cb607015 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:17.527534 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c7e986-bbdb-4f0b-9772-dff7cb607015 None None] PATH_INFO: `/` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:17.527534 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 72/142] 10.222.0.55 () {54 vars in 927 bytes} [Fri Sep 23 14:58:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 14:58:17.534561 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f2987b-2793-4bb7-97ef-c80dad6bfe3a None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:17.535105 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f2987b-2793-4bb7-97ef-c80dad6bfe3a None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:17.535594 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f2987b-2793-4bb7-97ef-c80dad6bfe3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:17.555751 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-99f2987b-2793-4bb7-97ef-c80dad6bfe3a None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 14:58:17.557424 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-99f2987b-2793-4bb7-97ef-c80dad6bfe3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 14:58:17.587604 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:17.631002 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 71/143] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Sep 23 14:58:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 14:58:17.913556 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c7e986-bbdb-4f0b-9772-dff7cb607015 None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:17.963606 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:18.056630 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c7e986-bbdb-4f0b-9772-dff7cb607015 None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:18.105495 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39998025-9920-4808-959b-82feaa7ad78d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:18.106484 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39998025-9920-4808-959b-82feaa7ad78d None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:18.106972 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39998025-9920-4808-959b-82feaa7ad78d None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:18.107461 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39998025-9920-4808-959b-82feaa7ad78d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:18.119443 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39998025-9920-4808-959b-82feaa7ad78d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 14:58:18.124570 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application [None req-39998025-9920-4808-959b-82feaa7ad78d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 23 14:58:18.124570 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 14:58:18.124570 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 14:58:18.124570 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 14:58:18.124570 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 14:58:18.124570 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 14:58:18.124570 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 14:58:18.124570 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 14:58:18.124570 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 14:58:18.124570 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 14:58:18.124570 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 14:58:18.124570 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 14:58:18.124570 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 14:58:18.124570 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 14:58:18.124570 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 14:58:18.124570 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 14:58:18.125450 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 14:58:18.125450 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 14:58:18.125450 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 14:58:18.125450 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 14:58:18.125450 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 14:58:18.125450 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 14:58:18.125450 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 14:58:18.125450 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 14:58:18.125450 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 14:58:18.125450 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 14:58:18.125450 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 14:58:18.125450 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application with Lock( Sep 23 14:58:18.125450 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 14:58:18.125450 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._enter() Sep 23 14:58:18.125450 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 14:58:18.125450 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 14:58:18.126224 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 14:58:18.126224 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.creator() Sep 23 14:58:18.126224 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 14:58:18.126224 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application created_value = creator( Sep 23 14:58:18.126224 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 14:58:18.126224 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 14:58:18.126224 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 14:58:18.126224 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 14:58:18.126224 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 14:58:18.126224 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 14:58:18.126224 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 23 14:58:18.126224 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Sep 23 14:58:18.128513 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 73/144] 10.222.0.55 () {56 vars in 1099 bytes} [Fri Sep 23 14:58:17 2022] GET /identity/v3/roles/reader => generated 84 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 14:58:18.134740 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f2987b-2793-4bb7-97ef-c80dad6bfe3a None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:18.238158 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f2987b-2793-4bb7-97ef-c80dad6bfe3a None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:18.275488 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505b22cc-b310-4d09-8dec-a4e93977bd1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:18.276702 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505b22cc-b310-4d09-8dec-a4e93977bd1f None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:18.277202 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505b22cc-b310-4d09-8dec-a4e93977bd1f None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:18.277693 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505b22cc-b310-4d09-8dec-a4e93977bd1f None admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:18.278568 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505b22cc-b310-4d09-8dec-a4e93977bd1f None admin] RBAC: Adding query filter params (name=reader) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:58:18.279076 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505b22cc-b310-4d09-8dec-a4e93977bd1f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:18.280698 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505b22cc-b310-4d09-8dec-a4e93977bd1f None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:18.291114 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 72/145] 10.222.0.55 () {56 vars in 1101 bytes} [Fri Sep 23 14:58:18 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:58:18.297292 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39998025-9920-4808-959b-82feaa7ad78d None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:18.402338 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:18.412696 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39998025-9920-4808-959b-82feaa7ad78d None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:18.459918 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2189f1dd-4005-4d6a-8892-cb4209a4c8ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:18.460957 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2189f1dd-4005-4d6a-8892-cb4209a4c8ce None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:18.461438 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2189f1dd-4005-4d6a-8892-cb4209a4c8ce None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:18.462559 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2189f1dd-4005-4d6a-8892-cb4209a4c8ce None admin] PATH_INFO: `/v3/users/fc9052ae566f4bd2ab726f74f7a156cd` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:18.463992 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2189f1dd-4005-4d6a-8892-cb4209a4c8ce None admin] RBAC: Authorizing `identity:get_user(user_id=fc9052ae566f4bd2ab726f74f7a156cd)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:18.465791 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2189f1dd-4005-4d6a-8892-cb4209a4c8ce None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:18.467653 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 74/146] 10.222.0.55 () {56 vars in 1177 bytes} [Fri Sep 23 14:58:18 2022] GET /identity/v3/users/fc9052ae566f4bd2ab726f74f7a156cd => generated 289 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:58:18.473650 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505b22cc-b310-4d09-8dec-a4e93977bd1f None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:18.512470 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:18.589067 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505b22cc-b310-4d09-8dec-a4e93977bd1f None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:18.624846 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e69f676-4122-426b-89c4-71949cfb0433 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:18.625866 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e69f676-4122-426b-89c4-71949cfb0433 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:18.626357 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e69f676-4122-426b-89c4-71949cfb0433 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:18.626816 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e69f676-4122-426b-89c4-71949cfb0433 None admin] PATH_INFO: `/v3/projects/0ffbe3ca86954853be08ecf627d6af00` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:18.628173 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e69f676-4122-426b-89c4-71949cfb0433 None admin] RBAC: Authorizing `identity:get_project(project_id=0ffbe3ca86954853be08ecf627d6af00)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:18.629804 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e69f676-4122-426b-89c4-71949cfb0433 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:18.631660 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 73/147] 10.222.0.55 () {56 vars in 1186 bytes} [Fri Sep 23 14:58:18 2022] GET /identity/v3/projects/0ffbe3ca86954853be08ecf627d6af00 => generated 298 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:58:18.638637 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2189f1dd-4005-4d6a-8892-cb4209a4c8ce None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:18.755096 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2189f1dd-4005-4d6a-8892-cb4209a4c8ce None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:18.799861 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8647a245-d130-4fc6-bb43-f1b2a2742fe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:18.800910 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8647a245-d130-4fc6-bb43-f1b2a2742fe4 None admin] REQUEST_METHOD: `PUT` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:18.801398 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8647a245-d130-4fc6-bb43-f1b2a2742fe4 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:18.801857 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8647a245-d130-4fc6-bb43-f1b2a2742fe4 None admin] PATH_INFO: `/v3/projects/0ffbe3ca86954853be08ecf627d6af00/users/fc9052ae566f4bd2ab726f74f7a156cd/roles/ea002840090d4194bb75fa083a55dc5f` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:18.804003 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8647a245-d130-4fc6-bb43-f1b2a2742fe4 None admin] RBAC: Authorizing `identity:create_grant(project_id=0ffbe3ca86954853be08ecf627d6af00, user_id=fc9052ae566f4bd2ab726f74f7a156cd, role_id=ea002840090d4194bb75fa083a55dc5f)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:18.806297 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8647a245-d130-4fc6-bb43-f1b2a2742fe4 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:18.905716 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 75/148] 10.222.0.55 () {58 vars in 1439 bytes} [Fri Sep 23 14:58:18 2022] PUT /identity/v3/projects/0ffbe3ca86954853be08ecf627d6af00/users/fc9052ae566f4bd2ab726f74f7a156cd/roles/ea002840090d4194bb75fa083a55dc5f => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 14:58:19.110397 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8aacfef-2e43-4063-9237-3198f66b443f None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:19.111033 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8aacfef-2e43-4063-9237-3198f66b443f None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:19.111507 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8aacfef-2e43-4063-9237-3198f66b443f None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:19.113209 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 74/149] 10.222.0.55 () {54 vars in 927 bytes} [Fri Sep 23 14:58:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 14:58:19.123067 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5658cc-a6b4-4816-b343-5824a36a3bc7 None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:19.123697 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5658cc-a6b4-4816-b343-5824a36a3bc7 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:19.124195 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5658cc-a6b4-4816-b343-5824a36a3bc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:19.148010 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-2d5658cc-a6b4-4816-b343-5824a36a3bc7 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 14:58:19.149655 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-2d5658cc-a6b4-4816-b343-5824a36a3bc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 14:58:19.171884 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:19.194668 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a583b92-2124-4ce3-80ba-25a7137deb40 None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:19.195011 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a583b92-2124-4ce3-80ba-25a7137deb40 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:19.195280 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a583b92-2124-4ce3-80ba-25a7137deb40 None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:19.197155 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 75/150] 10.222.0.55 () {54 vars in 927 bytes} [Fri Sep 23 14:58:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 23 14:58:19.204082 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb4b888-eb30-4110-aa23-1fc1046fb28a None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:19.204418 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb4b888-eb30-4110-aa23-1fc1046fb28a None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:19.204638 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb4b888-eb30-4110-aa23-1fc1046fb28a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:19.237823 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-bcb4b888-eb30-4110-aa23-1fc1046fb28a None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 14:58:19.239058 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-bcb4b888-eb30-4110-aa23-1fc1046fb28a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 14:58:19.264165 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 76/151] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Sep 23 14:58:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 14:58:19.355890 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 76/152] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Sep 23 14:58:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 14:58:19.422194 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ca7047-8b29-4c01-ad17-f6773dd345e6 None None] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:19.422526 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ca7047-8b29-4c01-ad17-f6773dd345e6 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:19.422753 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ca7047-8b29-4c01-ad17-f6773dd345e6 None None] PATH_INFO: `/` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:19.424377 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 77/153] 10.222.0.55 () {54 vars in 927 bytes} [Fri Sep 23 14:58:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 14:58:19.437974 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4e2d2a-c724-4caa-ba56-b7de7b6b8fcf None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:19.438291 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4e2d2a-c724-4caa-ba56-b7de7b6b8fcf None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:19.438533 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4e2d2a-c724-4caa-ba56-b7de7b6b8fcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:19.461040 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-cf4e2d2a-c724-4caa-ba56-b7de7b6b8fcf None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 14:58:19.462902 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-cf4e2d2a-c724-4caa-ba56-b7de7b6b8fcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 14:58:19.485190 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:19.550135 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 77/154] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Sep 23 14:58:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 14:58:19.778186 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ca7047-8b29-4c01-ad17-f6773dd345e6 None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:19.922022 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ca7047-8b29-4c01-ad17-f6773dd345e6 None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:19.958646 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4e2d2a-c724-4caa-ba56-b7de7b6b8fcf None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:19.972254 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5853a9-9de9-48de-b7c1-ee62b7d4377a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:19.972909 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5853a9-9de9-48de-b7c1-ee62b7d4377a None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:19.973181 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5853a9-9de9-48de-b7c1-ee62b7d4377a None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:19.973448 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5853a9-9de9-48de-b7c1-ee62b7d4377a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:19.998333 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5853a9-9de9-48de-b7c1-ee62b7d4377a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 14:58:20.005516 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application [None req-3b5853a9-9de9-48de-b7c1-ee62b7d4377a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 23 14:58:20.005516 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 14:58:20.005516 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 14:58:20.005516 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 14:58:20.005516 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 14:58:20.005516 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 14:58:20.005516 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 14:58:20.005516 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 14:58:20.005516 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 14:58:20.005516 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 14:58:20.005516 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 14:58:20.005516 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 14:58:20.005516 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 14:58:20.005516 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 14:58:20.005516 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 14:58:20.005516 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 14:58:20.006387 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 14:58:20.006387 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 14:58:20.006387 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 14:58:20.006387 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 14:58:20.006387 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 14:58:20.006387 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 14:58:20.006387 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 14:58:20.006387 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 14:58:20.006387 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 14:58:20.006387 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 14:58:20.006387 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 14:58:20.006387 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application with Lock( Sep 23 14:58:20.006387 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 14:58:20.006387 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._enter() Sep 23 14:58:20.006387 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 14:58:20.006387 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 14:58:20.007154 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 14:58:20.007154 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.creator() Sep 23 14:58:20.007154 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 14:58:20.007154 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application created_value = creator( Sep 23 14:58:20.007154 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 14:58:20.007154 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 14:58:20.007154 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 14:58:20.007154 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 14:58:20.007154 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 14:58:20.007154 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 14:58:20.007154 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 23 14:58:20.007154 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Sep 23 14:58:20.007154 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 78/155] 10.222.0.55 () {56 vars in 1096 bytes} [Fri Sep 23 14:58:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 14:58:20.045481 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5853a9-9de9-48de-b7c1-ee62b7d4377a None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:20.072514 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:20.214077 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4e2d2a-c724-4caa-ba56-b7de7b6b8fcf None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:20.219550 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5853a9-9de9-48de-b7c1-ee62b7d4377a None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:20.272397 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956ed75d-7685-4dcc-9da9-5f6c5f18e999 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:20.273448 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956ed75d-7685-4dcc-9da9-5f6c5f18e999 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:20.276355 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956ed75d-7685-4dcc-9da9-5f6c5f18e999 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:20.276812 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956ed75d-7685-4dcc-9da9-5f6c5f18e999 None admin] PATH_INFO: `/v3/roles` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:20.277661 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956ed75d-7685-4dcc-9da9-5f6c5f18e999 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:58:20.278187 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956ed75d-7685-4dcc-9da9-5f6c5f18e999 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:20.280404 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956ed75d-7685-4dcc-9da9-5f6c5f18e999 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:20.292520 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 79/156] 10.222.0.55 () {56 vars in 1099 bytes} [Fri Sep 23 14:58:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:58:20.294802 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956ed75d-7685-4dcc-9da9-5f6c5f18e999 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:20.359354 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8036f18-20ed-4578-bcac-37537497aab0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:20.360795 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8036f18-20ed-4578-bcac-37537497aab0 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:20.361221 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8036f18-20ed-4578-bcac-37537497aab0 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:20.361659 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8036f18-20ed-4578-bcac-37537497aab0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:20.395241 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8036f18-20ed-4578-bcac-37537497aab0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 14:58:20.409381 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-d8036f18-20ed-4578-bcac-37537497aab0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 23 14:58:20.409381 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 14:58:20.409381 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 14:58:20.409381 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 14:58:20.409381 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 14:58:20.409381 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 14:58:20.409381 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 14:58:20.409381 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 14:58:20.409381 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 14:58:20.409381 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 14:58:20.409381 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 14:58:20.409381 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 14:58:20.409381 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 14:58:20.409381 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 14:58:20.409381 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 14:58:20.409381 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 14:58:20.410591 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 14:58:20.410591 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 14:58:20.410591 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 14:58:20.410591 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 14:58:20.410591 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 14:58:20.410591 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 14:58:20.410591 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 14:58:20.410591 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 14:58:20.410591 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 14:58:20.410591 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 14:58:20.410591 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 14:58:20.410591 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Sep 23 14:58:20.410591 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 14:58:20.410591 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Sep 23 14:58:20.410591 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 14:58:20.410591 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 14:58:20.411548 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 14:58:20.411548 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Sep 23 14:58:20.411548 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 14:58:20.411548 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Sep 23 14:58:20.411548 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 14:58:20.411548 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 14:58:20.411548 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 14:58:20.411548 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 14:58:20.411548 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 14:58:20.411548 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 14:58:20.411548 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 23 14:58:20.411548 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Sep 23 14:58:20.411548 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 78/157] 10.222.0.55 () {56 vars in 1114 bytes} [Fri Sep 23 14:58:19 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 452 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 23 14:58:20.411548 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8036f18-20ed-4578-bcac-37537497aab0 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:20.432004 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:20.445205 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:20.480593 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956ed75d-7685-4dcc-9da9-5f6c5f18e999 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:20.551599 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ce3412-0be0-44c2-a7d4-9a2d087d50a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:20.552587 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ce3412-0be0-44c2-a7d4-9a2d087d50a5 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:20.553033 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ce3412-0be0-44c2-a7d4-9a2d087d50a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:20.553516 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ce3412-0be0-44c2-a7d4-9a2d087d50a5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:20.584705 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ce3412-0be0-44c2-a7d4-9a2d087d50a5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 14:58:20.591089 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application [None req-20ce3412-0be0-44c2-a7d4-9a2d087d50a5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 23 14:58:20.591089 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 14:58:20.591089 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 14:58:20.591089 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 14:58:20.591089 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 14:58:20.591089 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 14:58:20.591089 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 14:58:20.591089 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 14:58:20.591089 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 14:58:20.591089 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 14:58:20.591089 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 14:58:20.591089 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 14:58:20.591089 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 14:58:20.591089 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 14:58:20.591089 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 14:58:20.591089 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 14:58:20.592182 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 14:58:20.592182 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 14:58:20.592182 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 14:58:20.592182 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 14:58:20.592182 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 14:58:20.592182 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 14:58:20.592182 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 14:58:20.592182 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 14:58:20.592182 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 14:58:20.592182 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 14:58:20.592182 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 14:58:20.592182 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application with Lock( Sep 23 14:58:20.592182 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 14:58:20.592182 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._enter() Sep 23 14:58:20.592182 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 14:58:20.592182 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 14:58:20.593230 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 14:58:20.593230 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.creator() Sep 23 14:58:20.593230 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 14:58:20.593230 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application created_value = creator( Sep 23 14:58:20.593230 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 14:58:20.593230 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 14:58:20.593230 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 14:58:20.593230 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 14:58:20.593230 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 14:58:20.593230 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 14:58:20.593230 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 23 14:58:20.593230 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Sep 23 14:58:20.595494 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 80/158] 10.222.0.55 () {56 vars in 1099 bytes} [Fri Sep 23 14:58:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 14:58:20.600422 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ce3412-0be0-44c2-a7d4-9a2d087d50a5 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:20.651556 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8036f18-20ed-4578-bcac-37537497aab0 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:20.700539 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83bd47c-c8d4-468e-8a47-8ab8b2b0bfe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:20.701189 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83bd47c-c8d4-468e-8a47-8ab8b2b0bfe9 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:20.701465 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83bd47c-c8d4-468e-8a47-8ab8b2b0bfe9 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:20.701738 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83bd47c-c8d4-468e-8a47-8ab8b2b0bfe9 None admin] PATH_INFO: `/v3/users/0fc10e0211a54a0faf7ea9b86542de03` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:20.703060 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83bd47c-c8d4-468e-8a47-8ab8b2b0bfe9 None admin] RBAC: Authorizing `identity:get_user(user_id=0fc10e0211a54a0faf7ea9b86542de03)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:20.704671 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83bd47c-c8d4-468e-8a47-8ab8b2b0bfe9 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:20.706245 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 79/159] 10.222.0.55 () {56 vars in 1177 bytes} [Fri Sep 23 14:58:20 2022] GET /identity/v3/users/0fc10e0211a54a0faf7ea9b86542de03 => generated 247 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:58:20.707477 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83bd47c-c8d4-468e-8a47-8ab8b2b0bfe9 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:20.803499 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:20.808933 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ce3412-0be0-44c2-a7d4-9a2d087d50a5 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:20.825820 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83bd47c-c8d4-468e-8a47-8ab8b2b0bfe9 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:20.871886 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4057ece7-b165-4e3d-83ba-241a0b3abd04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:20.872936 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4057ece7-b165-4e3d-83ba-241a0b3abd04 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:20.873369 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4057ece7-b165-4e3d-83ba-241a0b3abd04 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:20.873921 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4057ece7-b165-4e3d-83ba-241a0b3abd04 None admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:20.874843 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4057ece7-b165-4e3d-83ba-241a0b3abd04 None admin] RBAC: Adding query filter params (name=member) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:58:20.875361 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4057ece7-b165-4e3d-83ba-241a0b3abd04 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:20.877205 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4057ece7-b165-4e3d-83ba-241a0b3abd04 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:20.888640 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 80/160] 10.222.0.55 () {56 vars in 1101 bytes} [Fri Sep 23 14:58:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:58:20.890438 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae174a7-1bf6-46b6-8720-1c96ca615cab None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:20.890945 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae174a7-1bf6-46b6-8720-1c96ca615cab None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:20.891420 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae174a7-1bf6-46b6-8720-1c96ca615cab None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:20.893412 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 81/161] 10.222.0.55 () {54 vars in 927 bytes} [Fri Sep 23 14:58:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 14:58:20.894368 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae174a7-1bf6-46b6-8720-1c96ca615cab None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:20.900766 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb94f3dc-9c94-4050-9d8a-9515b1d8787a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:20.901383 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb94f3dc-9c94-4050-9d8a-9515b1d8787a None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:20.901721 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb94f3dc-9c94-4050-9d8a-9515b1d8787a None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:20.901942 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb94f3dc-9c94-4050-9d8a-9515b1d8787a None admin] PATH_INFO: `/v3/roles` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:20.902625 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb94f3dc-9c94-4050-9d8a-9515b1d8787a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:58:20.902933 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb94f3dc-9c94-4050-9d8a-9515b1d8787a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:20.904372 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb94f3dc-9c94-4050-9d8a-9515b1d8787a None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:20.905982 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:20.920592 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 81/162] 10.222.0.55 () {56 vars in 1111 bytes} [Fri Sep 23 14:58:20 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:58:20.922848 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb94f3dc-9c94-4050-9d8a-9515b1d8787a None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:21.024598 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae174a7-1bf6-46b6-8720-1c96ca615cab None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:21.067022 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d250fa76-947a-44c0-ac4d-c039176c2e32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:21.068201 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d250fa76-947a-44c0-ac4d-c039176c2e32 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:21.068704 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d250fa76-947a-44c0-ac4d-c039176c2e32 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:21.069300 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d250fa76-947a-44c0-ac4d-c039176c2e32 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:21.093262 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:21.145943 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb94f3dc-9c94-4050-9d8a-9515b1d8787a None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:21.155501 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d250fa76-947a-44c0-ac4d-c039176c2e32 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:21.157632 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d250fa76-947a-44c0-ac4d-c039176c2e32 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:21.225257 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d402f6d7-73f1-4a04-aa27-ff0dbd88a069 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:21.225908 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d402f6d7-73f1-4a04-aa27-ff0dbd88a069 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:21.226193 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d402f6d7-73f1-4a04-aa27-ff0dbd88a069 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:21.226623 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d402f6d7-73f1-4a04-aa27-ff0dbd88a069 None admin] PATH_INFO: `/v3/users/ca9091e894bc4e01ab24bbbb474a09cd` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:21.232671 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d402f6d7-73f1-4a04-aa27-ff0dbd88a069 None admin] RBAC: Authorizing `identity:get_user(user_id=ca9091e894bc4e01ab24bbbb474a09cd)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:21.234169 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d402f6d7-73f1-4a04-aa27-ff0dbd88a069 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:21.235961 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 82/163] 10.222.0.55 () {56 vars in 1177 bytes} [Fri Sep 23 14:58:20 2022] GET /identity/v3/users/ca9091e894bc4e01ab24bbbb474a09cd => generated 275 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:58:21.236944 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f046249-cab9-4ce8-bd71-cb58c09942e8 None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:21.237230 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f046249-cab9-4ce8-bd71-cb58c09942e8 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:21.237531 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f046249-cab9-4ce8-bd71-cb58c09942e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:21.269224 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-5f046249-cab9-4ce8-bd71-cb58c09942e8 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 14:58:21.270442 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-5f046249-cab9-4ce8-bd71-cb58c09942e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 14:58:21.273311 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 82/164] 10.222.0.55 () {58 vars in 1282 bytes} [Fri Sep 23 14:58:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 23 14:58:21.274514 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d250fa76-947a-44c0-ac4d-c039176c2e32 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:21.333422 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:21.394629 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d250fa76-947a-44c0-ac4d-c039176c2e32 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:21.418032 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:21.437508 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c23babb-4c4b-4f39-9c1c-3df3adff87a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:21.438511 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c23babb-4c4b-4f39-9c1c-3df3adff87a1 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:21.439070 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c23babb-4c4b-4f39-9c1c-3df3adff87a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:21.439553 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c23babb-4c4b-4f39-9c1c-3df3adff87a1 None admin] PATH_INFO: `/v3/users/ca9091e894bc4e01ab24bbbb474a09cd` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:21.440932 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c23babb-4c4b-4f39-9c1c-3df3adff87a1 None admin] RBAC: Authorizing `identity:get_user(user_id=ca9091e894bc4e01ab24bbbb474a09cd)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:21.441650 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 83/165] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Sep 23 14:58:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 14:58:21.442575 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f046249-cab9-4ce8-bd71-cb58c09942e8 None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:21.443173 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c23babb-4c4b-4f39-9c1c-3df3adff87a1 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:21.445308 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 83/166] 10.222.0.55 () {56 vars in 1177 bytes} [Fri Sep 23 14:58:21 2022] GET /identity/v3/users/ca9091e894bc4e01ab24bbbb474a09cd => generated 275 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:58:21.447769 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c23babb-4c4b-4f39-9c1c-3df3adff87a1 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:21.578151 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c23babb-4c4b-4f39-9c1c-3df3adff87a1 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:21.612015 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:21.623061 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936a2c42-94d9-4847-a366-2def94afd563 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:21.627999 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936a2c42-94d9-4847-a366-2def94afd563 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:21.629863 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936a2c42-94d9-4847-a366-2def94afd563 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:21.629863 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936a2c42-94d9-4847-a366-2def94afd563 None admin] PATH_INFO: `/v3/projects/0ffbe3ca86954853be08ecf627d6af00` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:21.630646 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936a2c42-94d9-4847-a366-2def94afd563 None admin] RBAC: Authorizing `identity:get_project(project_id=0ffbe3ca86954853be08ecf627d6af00)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:21.632395 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936a2c42-94d9-4847-a366-2def94afd563 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:21.634470 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 84/167] 10.222.0.55 () {56 vars in 1186 bytes} [Fri Sep 23 14:58:21 2022] GET /identity/v3/projects/0ffbe3ca86954853be08ecf627d6af00 => generated 298 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:58:21.639834 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936a2c42-94d9-4847-a366-2def94afd563 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:21.753679 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f046249-cab9-4ce8-bd71-cb58c09942e8 None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:21.777394 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936a2c42-94d9-4847-a366-2def94afd563 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:21.825386 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52ed133-6779-4182-8ad1-1288a751917c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:21.826445 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52ed133-6779-4182-8ad1-1288a751917c None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:21.826940 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52ed133-6779-4182-8ad1-1288a751917c None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:21.827568 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfb1056-0d95-496f-8e97-627cf487b51d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:21.828149 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52ed133-6779-4182-8ad1-1288a751917c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:21.835999 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfb1056-0d95-496f-8e97-627cf487b51d None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:21.835999 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfb1056-0d95-496f-8e97-627cf487b51d None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:21.835999 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfb1056-0d95-496f-8e97-627cf487b51d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:21.938181 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:21.948334 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:21.953687 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfb1056-0d95-496f-8e97-627cf487b51d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:21.955190 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfb1056-0d95-496f-8e97-627cf487b51d None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:21.981611 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52ed133-6779-4182-8ad1-1288a751917c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:21.983076 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52ed133-6779-4182-8ad1-1288a751917c None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:22.065390 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 84/168] 10.222.0.55 () {58 vars in 1282 bytes} [Fri Sep 23 14:58:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 623 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 23 14:58:22.066180 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfb1056-0d95-496f-8e97-627cf487b51d None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:22.159863 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 85/169] 10.222.0.55 () {58 vars in 1282 bytes} [Fri Sep 23 14:58:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 520 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 23 14:58:22.160728 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52ed133-6779-4182-8ad1-1288a751917c None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:22.171361 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:22.177510 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfb1056-0d95-496f-8e97-627cf487b51d None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:22.215209 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e987e1-a5f7-4426-90eb-d66bff44779a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:22.215832 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e987e1-a5f7-4426-90eb-d66bff44779a None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:22.216159 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e987e1-a5f7-4426-90eb-d66bff44779a None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:22.216428 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e987e1-a5f7-4426-90eb-d66bff44779a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:22.217157 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e987e1-a5f7-4426-90eb-d66bff44779a None admin] RBAC: Adding query filter params (role.id=8e2919e79df040e8926f21607818b58c, scope.project.id=0ffbe3ca86954853be08ecf627d6af00, user.id=0fc10e0211a54a0faf7ea9b86542de03) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:58:22.217517 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e987e1-a5f7-4426-90eb-d66bff44779a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:22.218966 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e987e1-a5f7-4426-90eb-d66bff44779a None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:22.225967 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 85/170] 10.222.0.55 () {56 vars in 1374 bytes} [Fri Sep 23 14:58:22 2022] GET /identity/v3/role_assignments?user.id=0fc10e0211a54a0faf7ea9b86542de03&scope.project.id=0ffbe3ca86954853be08ecf627d6af00&role.id=8e2919e79df040e8926f21607818b58c => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:58:22.226808 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e987e1-a5f7-4426-90eb-d66bff44779a None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:22.344826 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e987e1-a5f7-4426-90eb-d66bff44779a None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:22.386479 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4fc880-4cea-4a64-94be-d79e7ec2b453 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:22.387136 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4fc880-4cea-4a64-94be-d79e7ec2b453 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:22.387416 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4fc880-4cea-4a64-94be-d79e7ec2b453 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:22.387690 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4fc880-4cea-4a64-94be-d79e7ec2b453 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:22.401652 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4fc880-4cea-4a64-94be-d79e7ec2b453 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 14:58:22.407812 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application [None req-9d4fc880-4cea-4a64-94be-d79e7ec2b453 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 23 14:58:22.407812 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 14:58:22.407812 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 14:58:22.407812 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 14:58:22.407812 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 14:58:22.407812 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 14:58:22.407812 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 14:58:22.407812 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 14:58:22.407812 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 14:58:22.407812 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 14:58:22.407812 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 14:58:22.407812 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 14:58:22.407812 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 14:58:22.407812 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 14:58:22.407812 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 14:58:22.407812 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 14:58:22.408749 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 14:58:22.408749 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 14:58:22.408749 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 14:58:22.408749 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 14:58:22.408749 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 14:58:22.408749 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 14:58:22.408749 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 14:58:22.408749 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 14:58:22.408749 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 14:58:22.408749 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 14:58:22.408749 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 14:58:22.408749 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application with Lock( Sep 23 14:58:22.408749 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 14:58:22.408749 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._enter() Sep 23 14:58:22.408749 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 14:58:22.408749 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 14:58:22.409537 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 14:58:22.409537 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.creator() Sep 23 14:58:22.409537 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 14:58:22.409537 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application created_value = creator( Sep 23 14:58:22.409537 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 14:58:22.409537 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 14:58:22.409537 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 14:58:22.409537 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 14:58:22.409537 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 14:58:22.409537 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 14:58:22.409537 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 23 14:58:22.409537 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Sep 23 14:58:22.409537 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 86/171] 10.222.0.55 () {56 vars in 1099 bytes} [Fri Sep 23 14:58:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 14:58:22.410267 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4fc880-4cea-4a64-94be-d79e7ec2b453 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:22.417593 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:22.436377 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:22.471384 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52ed133-6779-4182-8ad1-1288a751917c None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:22.553389 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edf976d-9075-4276-af68-52eac0dbd65a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:22.554057 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edf976d-9075-4276-af68-52eac0dbd65a None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:22.554338 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edf976d-9075-4276-af68-52eac0dbd65a None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:22.554610 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edf976d-9075-4276-af68-52eac0dbd65a None admin] PATH_INFO: `/v3/projects/0ffbe3ca86954853be08ecf627d6af00` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:22.556365 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edf976d-9075-4276-af68-52eac0dbd65a None admin] RBAC: Authorizing `identity:get_project(project_id=0ffbe3ca86954853be08ecf627d6af00)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:22.557809 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edf976d-9075-4276-af68-52eac0dbd65a None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:22.559441 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 86/172] 10.222.0.55 () {56 vars in 1186 bytes} [Fri Sep 23 14:58:22 2022] GET /identity/v3/projects/0ffbe3ca86954853be08ecf627d6af00 => generated 298 bytes in 399 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:58:22.560700 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f08e91-7139-419b-83f8-0ce33462a42b None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:22.560978 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f08e91-7139-419b-83f8-0ce33462a42b None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:22.561249 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f08e91-7139-419b-83f8-0ce33462a42b None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:22.561520 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4fc880-4cea-4a64-94be-d79e7ec2b453 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:22.562689 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 87/173] 10.222.0.55 () {54 vars in 927 bytes} [Fri Sep 23 14:58:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 14:58:22.563304 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f08e91-7139-419b-83f8-0ce33462a42b None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:22.604670 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9711e6-8b66-4f51-acf2-2ee00b923d39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:22.605247 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9711e6-8b66-4f51-acf2-2ee00b923d39 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:22.605537 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9711e6-8b66-4f51-acf2-2ee00b923d39 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:22.605816 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9711e6-8b66-4f51-acf2-2ee00b923d39 None admin] PATH_INFO: `/v3/projects/0ffbe3ca86954853be08ecf627d6af00` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:22.606972 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9711e6-8b66-4f51-acf2-2ee00b923d39 None admin] RBAC: Authorizing `identity:get_project(project_id=0ffbe3ca86954853be08ecf627d6af00)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:22.608596 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9711e6-8b66-4f51-acf2-2ee00b923d39 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:22.610388 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 87/174] 10.222.0.55 () {56 vars in 1186 bytes} [Fri Sep 23 14:58:22 2022] GET /identity/v3/projects/0ffbe3ca86954853be08ecf627d6af00 => generated 298 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:58:22.611353 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b276c63a-2849-48dd-b9d1-94db46ced7f2 None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:22.611630 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b276c63a-2849-48dd-b9d1-94db46ced7f2 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:22.611903 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b276c63a-2849-48dd-b9d1-94db46ced7f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:22.634525 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-b276c63a-2849-48dd-b9d1-94db46ced7f2 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 14:58:22.636057 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-b276c63a-2849-48dd-b9d1-94db46ced7f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 14:58:22.709683 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 88/175] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Sep 23 14:58:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 14:58:22.710377 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b276c63a-2849-48dd-b9d1-94db46ced7f2 None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:22.792393 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:22.795117 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f08e91-7139-419b-83f8-0ce33462a42b None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:22.855028 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe734d60-3630-46db-aa6e-aff6360a76cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:22.855631 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe734d60-3630-46db-aa6e-aff6360a76cb None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:22.855914 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe734d60-3630-46db-aa6e-aff6360a76cb None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:22.856207 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe734d60-3630-46db-aa6e-aff6360a76cb None admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:22.856889 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe734d60-3630-46db-aa6e-aff6360a76cb None admin] RBAC: Adding query filter params (name=member) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:58:22.857192 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe734d60-3630-46db-aa6e-aff6360a76cb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:22.857710 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b276c63a-2849-48dd-b9d1-94db46ced7f2 None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:22.858576 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe734d60-3630-46db-aa6e-aff6360a76cb None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:22.868264 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 88/176] 10.222.0.55 () {56 vars in 1101 bytes} [Fri Sep 23 14:58:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:58:22.869305 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe734d60-3630-46db-aa6e-aff6360a76cb None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:22.884552 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:22.897486 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adba4d0d-5c02-48da-84d9-09803c0f2830 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:22.898440 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adba4d0d-5c02-48da-84d9-09803c0f2830 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:22.898900 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adba4d0d-5c02-48da-84d9-09803c0f2830 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:22.899425 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adba4d0d-5c02-48da-84d9-09803c0f2830 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:22.900538 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adba4d0d-5c02-48da-84d9-09803c0f2830 None admin] RBAC: Adding query filter params (role.id=af257be2dee54d6b942f58650196582c, scope.project.id=0ffbe3ca86954853be08ecf627d6af00, user.id=ca9091e894bc4e01ab24bbbb474a09cd) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:58:22.901033 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adba4d0d-5c02-48da-84d9-09803c0f2830 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:22.916136 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adba4d0d-5c02-48da-84d9-09803c0f2830 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:22.916136 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 89/177] 10.222.0.55 () {56 vars in 1374 bytes} [Fri Sep 23 14:58:22 2022] GET /identity/v3/role_assignments?user.id=ca9091e894bc4e01ab24bbbb474a09cd&scope.project.id=0ffbe3ca86954853be08ecf627d6af00&role.id=af257be2dee54d6b942f58650196582c => generated 608 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:58:22.916136 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adba4d0d-5c02-48da-84d9-09803c0f2830 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:22.994739 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe734d60-3630-46db-aa6e-aff6360a76cb None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:23.036002 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8cbc69-5f49-4f30-9829-2055904b9a0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:23.036002 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8cbc69-5f49-4f30-9829-2055904b9a0e None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:23.036002 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8cbc69-5f49-4f30-9829-2055904b9a0e None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:23.036002 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8cbc69-5f49-4f30-9829-2055904b9a0e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:23.036544 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8cbc69-5f49-4f30-9829-2055904b9a0e None admin] RBAC: Adding query filter params (role.id=5c74fdc0e9cc44538cd8282006ed4443, scope.project.id=0ffbe3ca86954853be08ecf627d6af00, user.id=ca9091e894bc4e01ab24bbbb474a09cd) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:58:23.036544 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8cbc69-5f49-4f30-9829-2055904b9a0e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:23.036683 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8cbc69-5f49-4f30-9829-2055904b9a0e None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:23.043696 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 89/178] 10.222.0.55 () {56 vars in 1374 bytes} [Fri Sep 23 14:58:22 2022] GET /identity/v3/role_assignments?user.id=ca9091e894bc4e01ab24bbbb474a09cd&scope.project.id=0ffbe3ca86954853be08ecf627d6af00&role.id=5c74fdc0e9cc44538cd8282006ed4443 => generated 608 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:58:23.058491 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41b685b-ddb7-4f61-a7be-a743031b8143 None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:23.058765 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41b685b-ddb7-4f61-a7be-a743031b8143 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:23.059108 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41b685b-ddb7-4f61-a7be-a743031b8143 None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:23.065844 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:23.066090 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 90/179] 10.222.0.55 () {54 vars in 927 bytes} [Fri Sep 23 14:58:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 14:58:23.067672 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb95c9a-c5b0-49dd-982a-93ec4d23d9ac None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:23.067963 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb95c9a-c5b0-49dd-982a-93ec4d23d9ac None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:23.068266 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb95c9a-c5b0-49dd-982a-93ec4d23d9ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:23.091840 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adba4d0d-5c02-48da-84d9-09803c0f2830 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:23.102244 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-ffb95c9a-c5b0-49dd-982a-93ec4d23d9ac None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 14:58:23.103508 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-ffb95c9a-c5b0-49dd-982a-93ec4d23d9ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 14:58:23.131679 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dc3c96-1e2e-452b-ad36-fdc8308e2dbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:23.132761 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dc3c96-1e2e-452b-ad36-fdc8308e2dbc None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:23.133222 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dc3c96-1e2e-452b-ad36-fdc8308e2dbc None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:23.133675 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dc3c96-1e2e-452b-ad36-fdc8308e2dbc None admin] PATH_INFO: `/v3/users/ca9091e894bc4e01ab24bbbb474a09cd` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:23.135353 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7dc3c96-1e2e-452b-ad36-fdc8308e2dbc None admin] RBAC: Authorizing `identity:get_user(user_id=ca9091e894bc4e01ab24bbbb474a09cd)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:23.137177 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7dc3c96-1e2e-452b-ad36-fdc8308e2dbc None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:23.139154 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 90/180] 10.222.0.55 () {56 vars in 1177 bytes} [Fri Sep 23 14:58:22 2022] GET /identity/v3/users/ca9091e894bc4e01ab24bbbb474a09cd => generated 275 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:58:23.150015 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dc3c96-1e2e-452b-ad36-fdc8308e2dbc None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:23.233898 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 91/181] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Sep 23 14:58:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 14:58:23.234807 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb95c9a-c5b0-49dd-982a-93ec4d23d9ac None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:23.244399 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:23.377007 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb95c9a-c5b0-49dd-982a-93ec4d23d9ac None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:23.411652 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dc3c96-1e2e-452b-ad36-fdc8308e2dbc None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:23.420011 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c6d0ca-7578-4abb-8137-f1b6e1961ee4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:23.420011 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c6d0ca-7578-4abb-8137-f1b6e1961ee4 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:23.420011 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c6d0ca-7578-4abb-8137-f1b6e1961ee4 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:23.420011 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c6d0ca-7578-4abb-8137-f1b6e1961ee4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:23.429710 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c6d0ca-7578-4abb-8137-f1b6e1961ee4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 14:58:23.439220 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-48c6d0ca-7578-4abb-8137-f1b6e1961ee4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 23 14:58:23.439220 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 14:58:23.439220 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 14:58:23.439220 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 14:58:23.439220 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 14:58:23.439220 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 14:58:23.439220 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 14:58:23.439220 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 14:58:23.439220 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 14:58:23.439220 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 14:58:23.439220 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 14:58:23.439220 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 14:58:23.439220 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 14:58:23.439220 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 14:58:23.439220 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 14:58:23.439220 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 14:58:23.440209 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 14:58:23.440209 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 14:58:23.440209 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 14:58:23.440209 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 14:58:23.440209 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 14:58:23.440209 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 14:58:23.440209 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 14:58:23.440209 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 14:58:23.440209 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 14:58:23.440209 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 14:58:23.440209 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 14:58:23.440209 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Sep 23 14:58:23.440209 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 14:58:23.440209 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Sep 23 14:58:23.440209 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 14:58:23.440209 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 14:58:23.440977 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 14:58:23.440977 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Sep 23 14:58:23.440977 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 14:58:23.440977 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Sep 23 14:58:23.440977 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 14:58:23.440977 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 14:58:23.440977 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 14:58:23.440977 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 14:58:23.440977 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 14:58:23.440977 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 14:58:23.440977 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 23 14:58:23.440977 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Sep 23 14:58:23.440977 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 92/182] 10.222.0.55 () {56 vars in 1096 bytes} [Fri Sep 23 14:58:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 14:58:23.440977 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44931fc-df10-4eb7-92b0-0d5dd43eac45 None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:23.441900 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44931fc-df10-4eb7-92b0-0d5dd43eac45 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:23.441900 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44931fc-df10-4eb7-92b0-0d5dd43eac45 None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:23.441900 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 93/183] 10.222.0.55 () {54 vars in 927 bytes} [Fri Sep 23 14:58:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 14:58:23.441900 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b05c1da-1f20-4b3e-9ca5-7c89b94a5b4d None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:23.442140 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b05c1da-1f20-4b3e-9ca5-7c89b94a5b4d None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:23.442853 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b05c1da-1f20-4b3e-9ca5-7c89b94a5b4d None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:23.443519 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 94/184] 10.222.0.55 () {54 vars in 927 bytes} [Fri Sep 23 14:58:23 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 14:58:23.444331 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b05c1da-1f20-4b3e-9ca5-7c89b94a5b4d None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:23.467240 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17692259-3613-4b5c-9d1d-e164eba86be2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:23.468315 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17692259-3613-4b5c-9d1d-e164eba86be2 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:23.468801 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17692259-3613-4b5c-9d1d-e164eba86be2 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:23.469272 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17692259-3613-4b5c-9d1d-e164eba86be2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:23.489541 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:23.518538 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:23.556431 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17692259-3613-4b5c-9d1d-e164eba86be2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:23.558065 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17692259-3613-4b5c-9d1d-e164eba86be2 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:23.566600 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b05c1da-1f20-4b3e-9ca5-7c89b94a5b4d None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:23.606400 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb1bdcd-4e4a-4be1-9245-04b35924c3d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:23.607147 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb1bdcd-4e4a-4be1-9245-04b35924c3d6 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:23.607430 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb1bdcd-4e4a-4be1-9245-04b35924c3d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:23.607704 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb1bdcd-4e4a-4be1-9245-04b35924c3d6 None admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:23.608403 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb1bdcd-4e4a-4be1-9245-04b35924c3d6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:58:23.608695 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb1bdcd-4e4a-4be1-9245-04b35924c3d6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:23.610061 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb1bdcd-4e4a-4be1-9245-04b35924c3d6 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:23.619065 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 95/185] 10.222.0.55 () {56 vars in 1099 bytes} [Fri Sep 23 14:58:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:58:23.620403 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d1876f-4fcc-4543-8f8a-345c211b4101 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:23.620731 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d1876f-4fcc-4543-8f8a-345c211b4101 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:23.621010 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d1876f-4fcc-4543-8f8a-345c211b4101 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:23.639019 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-75d1876f-4fcc-4543-8f8a-345c211b4101 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 14:58:23.640233 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-75d1876f-4fcc-4543-8f8a-345c211b4101 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 14:58:23.663074 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 91/186] 10.222.0.55 () {58 vars in 1282 bytes} [Fri Sep 23 14:58:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 514 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 23 14:58:23.664283 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12015f7-f238-4280-8f6d-2dbf759143fc None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:23.664554 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12015f7-f238-4280-8f6d-2dbf759143fc None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:23.664828 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12015f7-f238-4280-8f6d-2dbf759143fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:23.685263 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-a12015f7-f238-4280-8f6d-2dbf759143fc None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 14:58:23.698421 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-a12015f7-f238-4280-8f6d-2dbf759143fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 14:58:23.709524 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 96/187] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Sep 23 14:58:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 14:58:23.716072 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d1876f-4fcc-4543-8f8a-345c211b4101 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:23.731252 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:23.754028 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 92/188] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Sep 23 14:58:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 14:58:23.754869 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12015f7-f238-4280-8f6d-2dbf759143fc None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:23.786791 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:23.843447 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d1876f-4fcc-4543-8f8a-345c211b4101 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:23.860804 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12015f7-f238-4280-8f6d-2dbf759143fc None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:23.888104 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c903fe2a-4353-4530-9f6b-2452c18eacdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:23.889129 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c903fe2a-4353-4530-9f6b-2452c18eacdc None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:23.889666 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c903fe2a-4353-4530-9f6b-2452c18eacdc None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:23.890284 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c903fe2a-4353-4530-9f6b-2452c18eacdc None admin] PATH_INFO: `/v3/users/755ef177078341b6a1f30e126cb4a9fa` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:23.901490 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ceea299-cacc-41d9-a31b-d24ce309f8be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:23.902504 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ceea299-cacc-41d9-a31b-d24ce309f8be None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:23.902969 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ceea299-cacc-41d9-a31b-d24ce309f8be None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:23.903421 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ceea299-cacc-41d9-a31b-d24ce309f8be None admin] PATH_INFO: `/v3/domains/default` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:23.904885 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ceea299-cacc-41d9-a31b-d24ce309f8be None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:23.906338 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ceea299-cacc-41d9-a31b-d24ce309f8be None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:23.908182 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 93/189] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Sep 23 14:58:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:58:23.908926 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ceea299-cacc-41d9-a31b-d24ce309f8be None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:23.909661 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c903fe2a-4353-4530-9f6b-2452c18eacdc None admin] RBAC: Authorizing `identity:get_user(user_id=755ef177078341b6a1f30e126cb4a9fa)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:23.911684 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c903fe2a-4353-4530-9f6b-2452c18eacdc None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:23.914588 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 97/190] 10.222.0.55 () {56 vars in 1177 bytes} [Fri Sep 23 14:58:23 2022] GET /identity/v3/users/755ef177078341b6a1f30e126cb4a9fa => generated 283 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:58:23.916482 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c903fe2a-4353-4530-9f6b-2452c18eacdc None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:23.966090 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:24.021278 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ceea299-cacc-41d9-a31b-d24ce309f8be None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:24.043622 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c903fe2a-4353-4530-9f6b-2452c18eacdc None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:24.057987 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7399b476-d3a8-45ff-be94-62550d7fb790 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:24.058632 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7399b476-d3a8-45ff-be94-62550d7fb790 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:24.058892 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:24.059016 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7399b476-d3a8-45ff-be94-62550d7fb790 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:24.059168 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7399b476-d3a8-45ff-be94-62550d7fb790 None admin] PATH_INFO: `/v3/projects/8381c67dd5274d1780bb20df47b105ec` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:24.060358 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7399b476-d3a8-45ff-be94-62550d7fb790 None admin] RBAC: Authorizing `identity:get_project(project_id=8381c67dd5274d1780bb20df47b105ec)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:24.061803 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7399b476-d3a8-45ff-be94-62550d7fb790 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:24.063638 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 94/191] 10.222.0.55 () {56 vars in 1186 bytes} [Fri Sep 23 14:58:23 2022] GET /identity/v3/projects/8381c67dd5274d1780bb20df47b105ec => generated 312 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:58:24.064231 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7399b476-d3a8-45ff-be94-62550d7fb790 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:24.084276 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5300d2-7c59-42fe-a580-9a28ab972f77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:24.084890 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5300d2-7c59-42fe-a580-9a28ab972f77 None admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:24.085166 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5300d2-7c59-42fe-a580-9a28ab972f77 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:24.085433 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5300d2-7c59-42fe-a580-9a28ab972f77 None admin] PATH_INFO: `/v3/users` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:24.086077 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5300d2-7c59-42fe-a580-9a28ab972f77 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:24.087662 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5300d2-7c59-42fe-a580-9a28ab972f77 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:24.170383 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7399b476-d3a8-45ff-be94-62550d7fb790 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:24.183223 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 98/192] 10.222.0.55 () {60 vars in 1132 bytes} [Fri Sep 23 14:58:23 2022] POST /identity/v3/users => generated 297 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 14:58:24.184105 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5300d2-7c59-42fe-a580-9a28ab972f77 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:24.193871 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:24.218133 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b6bb5a-0aa8-444e-a7a1-06e8d54340fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:24.218919 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b6bb5a-0aa8-444e-a7a1-06e8d54340fb None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:24.219208 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b6bb5a-0aa8-444e-a7a1-06e8d54340fb None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:24.219482 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b6bb5a-0aa8-444e-a7a1-06e8d54340fb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:24.297186 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b6bb5a-0aa8-444e-a7a1-06e8d54340fb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:24.298653 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b6bb5a-0aa8-444e-a7a1-06e8d54340fb None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:24.303760 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5300d2-7c59-42fe-a580-9a28ab972f77 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:24.352330 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74202c39-2976-4eeb-975d-a69d2e1f7ef9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:24.353682 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74202c39-2976-4eeb-975d-a69d2e1f7ef9 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:24.353968 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74202c39-2976-4eeb-975d-a69d2e1f7ef9 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:24.354252 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74202c39-2976-4eeb-975d-a69d2e1f7ef9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:24.369191 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74202c39-2976-4eeb-975d-a69d2e1f7ef9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 14:58:24.374909 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-74202c39-2976-4eeb-975d-a69d2e1f7ef9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 23 14:58:24.374909 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 14:58:24.374909 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 14:58:24.374909 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 14:58:24.374909 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 14:58:24.374909 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 14:58:24.374909 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 14:58:24.374909 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 14:58:24.374909 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 14:58:24.374909 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 14:58:24.374909 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 14:58:24.374909 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 14:58:24.374909 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 14:58:24.374909 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 14:58:24.374909 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 14:58:24.374909 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 14:58:24.375814 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 14:58:24.375814 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 14:58:24.375814 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 14:58:24.375814 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 14:58:24.375814 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 14:58:24.375814 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 14:58:24.375814 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 14:58:24.375814 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 14:58:24.375814 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 14:58:24.375814 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 14:58:24.375814 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 14:58:24.375814 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Sep 23 14:58:24.375814 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 14:58:24.375814 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Sep 23 14:58:24.375814 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 14:58:24.375814 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 14:58:24.376938 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 14:58:24.376938 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Sep 23 14:58:24.376938 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 14:58:24.376938 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Sep 23 14:58:24.376938 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 14:58:24.376938 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 14:58:24.376938 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 14:58:24.376938 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 14:58:24.376938 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 14:58:24.376938 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 14:58:24.376938 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 23 14:58:24.376938 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Sep 23 14:58:24.376938 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 99/193] 10.222.0.55 () {56 vars in 1114 bytes} [Fri Sep 23 14:58:24 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 14:58:24.377708 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74202c39-2976-4eeb-975d-a69d2e1f7ef9 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:24.420542 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:24.429751 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 95/194] 10.222.0.55 () {58 vars in 1282 bytes} [Fri Sep 23 14:58:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 23 14:58:24.430514 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b6bb5a-0aa8-444e-a7a1-06e8d54340fb None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:24.452861 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:24.506947 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74202c39-2976-4eeb-975d-a69d2e1f7ef9 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:24.559870 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62df58e-dd6d-4f14-bfa2-7a3a70ac0bd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:24.560557 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62df58e-dd6d-4f14-bfa2-7a3a70ac0bd0 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:24.560838 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62df58e-dd6d-4f14-bfa2-7a3a70ac0bd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:24.561111 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62df58e-dd6d-4f14-bfa2-7a3a70ac0bd0 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:24.574244 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62df58e-dd6d-4f14-bfa2-7a3a70ac0bd0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 14:58:24.577194 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b6bb5a-0aa8-444e-a7a1-06e8d54340fb None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:24.579611 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-c62df58e-dd6d-4f14-bfa2-7a3a70ac0bd0 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 23 14:58:24.579611 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 14:58:24.579611 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 14:58:24.579611 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 14:58:24.579611 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 14:58:24.579611 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 14:58:24.579611 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 14:58:24.579611 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 14:58:24.579611 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 14:58:24.579611 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 14:58:24.579611 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 14:58:24.579611 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 14:58:24.579611 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 14:58:24.579611 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 14:58:24.579611 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 14:58:24.579611 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 14:58:24.580580 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 14:58:24.580580 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 14:58:24.580580 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 14:58:24.580580 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 14:58:24.580580 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 14:58:24.580580 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 14:58:24.580580 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 14:58:24.580580 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 14:58:24.580580 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 14:58:24.580580 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 14:58:24.580580 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 14:58:24.580580 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Sep 23 14:58:24.580580 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 14:58:24.580580 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Sep 23 14:58:24.580580 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 14:58:24.580580 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 14:58:24.581375 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 14:58:24.581375 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Sep 23 14:58:24.581375 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 14:58:24.581375 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Sep 23 14:58:24.581375 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 14:58:24.581375 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 14:58:24.581375 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 14:58:24.581375 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 14:58:24.581375 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 14:58:24.581375 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 14:58:24.581375 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 23 14:58:24.581375 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Sep 23 14:58:24.581375 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 100/195] 10.222.0.55 () {56 vars in 1099 bytes} [Fri Sep 23 14:58:24 2022] GET /identity/v3/roles/reader => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 14:58:24.582030 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62df58e-dd6d-4f14-bfa2-7a3a70ac0bd0 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:24.624454 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b056b2-17e9-4fbc-b297-01941c7275a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:24.625486 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b056b2-17e9-4fbc-b297-01941c7275a2 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:24.626759 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b056b2-17e9-4fbc-b297-01941c7275a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:24.627190 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b056b2-17e9-4fbc-b297-01941c7275a2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:24.628126 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b056b2-17e9-4fbc-b297-01941c7275a2 None admin] RBAC: Adding query filter params (role.id=af257be2dee54d6b942f58650196582c, scope.project.id=8381c67dd5274d1780bb20df47b105ec, user.id=ca9091e894bc4e01ab24bbbb474a09cd) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:58:24.628582 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b056b2-17e9-4fbc-b297-01941c7275a2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:24.630199 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b056b2-17e9-4fbc-b297-01941c7275a2 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:24.638362 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 96/196] 10.222.0.55 () {56 vars in 1374 bytes} [Fri Sep 23 14:58:24 2022] GET /identity/v3/role_assignments?user.id=ca9091e894bc4e01ab24bbbb474a09cd&scope.project.id=8381c67dd5274d1780bb20df47b105ec&role.id=af257be2dee54d6b942f58650196582c => generated 608 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:58:24.639380 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b056b2-17e9-4fbc-b297-01941c7275a2 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:24.741357 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:24.741563 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:24.763136 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62df58e-dd6d-4f14-bfa2-7a3a70ac0bd0 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:24.768840 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b056b2-17e9-4fbc-b297-01941c7275a2 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:24.815428 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6c15bc-ec06-4a6d-9299-d174d06b1f36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:24.816117 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6c15bc-ec06-4a6d-9299-d174d06b1f36 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:24.816406 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6c15bc-ec06-4a6d-9299-d174d06b1f36 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:24.816682 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6c15bc-ec06-4a6d-9299-d174d06b1f36 None admin] PATH_INFO: `/v3/projects/346756b911774d55af2a9ad3de2143b4` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:24.818177 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6c15bc-ec06-4a6d-9299-d174d06b1f36 None admin] RBAC: Authorizing `identity:get_project(project_id=346756b911774d55af2a9ad3de2143b4)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:24.819845 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6c15bc-ec06-4a6d-9299-d174d06b1f36 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:24.822616 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 97/197] 10.222.0.55 () {56 vars in 1186 bytes} [Fri Sep 23 14:58:24 2022] GET /identity/v3/projects/346756b911774d55af2a9ad3de2143b4 => generated 302 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:58:24.823932 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6c15bc-ec06-4a6d-9299-d174d06b1f36 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:24.838158 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d219b26-b0b1-463e-867a-a43c97a4575d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:24.838851 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d219b26-b0b1-463e-867a-a43c97a4575d None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:24.839129 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d219b26-b0b1-463e-867a-a43c97a4575d None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:24.839396 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d219b26-b0b1-463e-867a-a43c97a4575d None admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:24.840104 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d219b26-b0b1-463e-867a-a43c97a4575d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:58:24.840464 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d219b26-b0b1-463e-867a-a43c97a4575d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:24.841912 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d219b26-b0b1-463e-867a-a43c97a4575d None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:24.850820 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 101/198] 10.222.0.55 () {56 vars in 1111 bytes} [Fri Sep 23 14:58:24 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:58:24.851571 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d219b26-b0b1-463e-867a-a43c97a4575d None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:24.977000 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6c15bc-ec06-4a6d-9299-d174d06b1f36 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:25.026104 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d219b26-b0b1-463e-867a-a43c97a4575d None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:25.028061 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe6c753-9911-41d0-87a1-1b34aa5d01eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:25.028687 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe6c753-9911-41d0-87a1-1b34aa5d01eb None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:25.028964 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe6c753-9911-41d0-87a1-1b34aa5d01eb None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:25.029232 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe6c753-9911-41d0-87a1-1b34aa5d01eb None admin] PATH_INFO: `/v3/roles` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:25.030477 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe6c753-9911-41d0-87a1-1b34aa5d01eb None admin] RBAC: Adding query filter params (name=reader) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:58:25.030477 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe6c753-9911-41d0-87a1-1b34aa5d01eb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:25.031623 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe6c753-9911-41d0-87a1-1b34aa5d01eb None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:25.042769 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 98/199] 10.222.0.55 () {56 vars in 1101 bytes} [Fri Sep 23 14:58:24 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:58:25.043704 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe6c753-9911-41d0-87a1-1b34aa5d01eb None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:25.056453 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:25.071755 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3510ff4-b923-48f3-981a-f8453ada2c2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:25.072769 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3510ff4-b923-48f3-981a-f8453ada2c2d None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:25.073249 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3510ff4-b923-48f3-981a-f8453ada2c2d None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:25.073816 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3510ff4-b923-48f3-981a-f8453ada2c2d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:25.075226 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3510ff4-b923-48f3-981a-f8453ada2c2d None admin] RBAC: Adding query filter params (role.id=8e2919e79df040e8926f21607818b58c, scope.project.id=346756b911774d55af2a9ad3de2143b4, user.id=755ef177078341b6a1f30e126cb4a9fa) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:58:25.076441 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3510ff4-b923-48f3-981a-f8453ada2c2d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:25.078182 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3510ff4-b923-48f3-981a-f8453ada2c2d None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:25.087747 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 102/200] 10.222.0.55 () {56 vars in 1374 bytes} [Fri Sep 23 14:58:24 2022] GET /identity/v3/role_assignments?user.id=755ef177078341b6a1f30e126cb4a9fa&scope.project.id=346756b911774d55af2a9ad3de2143b4&role.id=8e2919e79df040e8926f21607818b58c => generated 259 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:58:25.088878 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3510ff4-b923-48f3-981a-f8453ada2c2d None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:25.136941 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:25.251841 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe6c753-9911-41d0-87a1-1b34aa5d01eb None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:25.317577 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3510ff4-b923-48f3-981a-f8453ada2c2d None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:25.319863 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d50f072-bf31-4c23-85d4-f29ff3fee8e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:25.320944 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d50f072-bf31-4c23-85d4-f29ff3fee8e0 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:25.321468 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d50f072-bf31-4c23-85d4-f29ff3fee8e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:25.321937 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d50f072-bf31-4c23-85d4-f29ff3fee8e0 None admin] PATH_INFO: `/v3/users/755ef177078341b6a1f30e126cb4a9fa` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:25.325394 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d50f072-bf31-4c23-85d4-f29ff3fee8e0 None admin] RBAC: Authorizing `identity:get_user(user_id=755ef177078341b6a1f30e126cb4a9fa)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:25.327510 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d50f072-bf31-4c23-85d4-f29ff3fee8e0 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:25.329839 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 99/201] 10.222.0.55 () {56 vars in 1177 bytes} [Fri Sep 23 14:58:25 2022] GET /identity/v3/users/755ef177078341b6a1f30e126cb4a9fa => generated 283 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:58:25.335386 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d50f072-bf31-4c23-85d4-f29ff3fee8e0 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:25.371895 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f069fa90-c7e6-4e56-bced-8a97dea02886 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:25.372577 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f069fa90-c7e6-4e56-bced-8a97dea02886 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:25.372857 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f069fa90-c7e6-4e56-bced-8a97dea02886 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:25.373128 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f069fa90-c7e6-4e56-bced-8a97dea02886 None admin] PATH_INFO: `/v3/users/fc9052ae566f4bd2ab726f74f7a156cd` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:25.374380 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f069fa90-c7e6-4e56-bced-8a97dea02886 None admin] RBAC: Authorizing `identity:get_user(user_id=fc9052ae566f4bd2ab726f74f7a156cd)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:25.380687 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f069fa90-c7e6-4e56-bced-8a97dea02886 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:25.382666 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 103/202] 10.222.0.55 () {56 vars in 1177 bytes} [Fri Sep 23 14:58:25 2022] GET /identity/v3/users/fc9052ae566f4bd2ab726f74f7a156cd => generated 289 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:58:25.388714 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f069fa90-c7e6-4e56-bced-8a97dea02886 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:25.437938 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:25.465855 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d50f072-bf31-4c23-85d4-f29ff3fee8e0 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:25.507537 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6630c678-977b-4e3e-9b79-24920fad74a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:25.508559 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6630c678-977b-4e3e-9b79-24920fad74a0 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:25.509266 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6630c678-977b-4e3e-9b79-24920fad74a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:25.509755 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6630c678-977b-4e3e-9b79-24920fad74a0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:25.590368 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6630c678-977b-4e3e-9b79-24920fad74a0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:25.592167 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6630c678-977b-4e3e-9b79-24920fad74a0 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:25.597077 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:25.639932 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f069fa90-c7e6-4e56-bced-8a97dea02886 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:25.687802 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:25.697882 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb1eaa1-fe74-41a4-b2b1-fb9cf944bd29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:25.698692 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb1eaa1-fe74-41a4-b2b1-fb9cf944bd29 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:25.699011 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb1eaa1-fe74-41a4-b2b1-fb9cf944bd29 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:25.699357 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb1eaa1-fe74-41a4-b2b1-fb9cf944bd29 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:25.721189 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 100/203] 10.222.0.55 () {58 vars in 1282 bytes} [Fri Sep 23 14:58:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 23 14:58:25.729667 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6630c678-977b-4e3e-9b79-24920fad74a0 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:25.819688 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb1eaa1-fe74-41a4-b2b1-fb9cf944bd29 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:25.821775 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb1eaa1-fe74-41a4-b2b1-fb9cf944bd29 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:25.897441 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6630c678-977b-4e3e-9b79-24920fad74a0 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:25.939744 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b13b18-3e34-452b-8560-e9940726e4ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:25.940994 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b13b18-3e34-452b-8560-e9940726e4ff None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:25.941428 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b13b18-3e34-452b-8560-e9940726e4ff None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:25.942225 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b13b18-3e34-452b-8560-e9940726e4ff None admin] PATH_INFO: `/v3/projects/346756b911774d55af2a9ad3de2143b4` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:25.943752 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b13b18-3e34-452b-8560-e9940726e4ff None admin] RBAC: Authorizing `identity:get_project(project_id=346756b911774d55af2a9ad3de2143b4)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:25.946793 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b13b18-3e34-452b-8560-e9940726e4ff None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:25.948582 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 101/204] 10.222.0.55 () {56 vars in 1186 bytes} [Fri Sep 23 14:58:25 2022] GET /identity/v3/projects/346756b911774d55af2a9ad3de2143b4 => generated 302 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 14:58:25.956937 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b13b18-3e34-452b-8560-e9940726e4ff None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:25.985568 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:26.011436 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 104/205] 10.222.0.55 () {58 vars in 1282 bytes} [Fri Sep 23 14:58:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 623 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 23 14:58:26.016970 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb1eaa1-fe74-41a4-b2b1-fb9cf944bd29 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:26.040557 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:26.138448 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b13b18-3e34-452b-8560-e9940726e4ff None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:26.204401 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40a089a-f88c-4561-b737-b2867edfc840 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:26.205064 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40a089a-f88c-4561-b737-b2867edfc840 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:26.205360 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40a089a-f88c-4561-b737-b2867edfc840 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:26.205626 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40a089a-f88c-4561-b737-b2867edfc840 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:26.206370 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40a089a-f88c-4561-b737-b2867edfc840 None admin] RBAC: Adding query filter params (role.id=5c74fdc0e9cc44538cd8282006ed4443, scope.project.id=346756b911774d55af2a9ad3de2143b4, user.id=755ef177078341b6a1f30e126cb4a9fa) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:58:26.206680 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40a089a-f88c-4561-b737-b2867edfc840 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:26.208170 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40a089a-f88c-4561-b737-b2867edfc840 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:26.212051 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb1eaa1-fe74-41a4-b2b1-fb9cf944bd29 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:26.223830 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 102/206] 10.222.0.55 () {56 vars in 1374 bytes} [Fri Sep 23 14:58:25 2022] GET /identity/v3/role_assignments?user.id=755ef177078341b6a1f30e126cb4a9fa&scope.project.id=346756b911774d55af2a9ad3de2143b4&role.id=5c74fdc0e9cc44538cd8282006ed4443 => generated 259 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 14:58:26.277467 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598081ff-099c-4aef-bf17-e2e633a9483f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:26.278130 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598081ff-099c-4aef-bf17-e2e633a9483f None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:26.278417 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598081ff-099c-4aef-bf17-e2e633a9483f None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:26.278696 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598081ff-099c-4aef-bf17-e2e633a9483f None admin] PATH_INFO: `/v3/projects/0ffbe3ca86954853be08ecf627d6af00` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:26.284966 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598081ff-099c-4aef-bf17-e2e633a9483f None admin] RBAC: Authorizing `identity:get_project(project_id=0ffbe3ca86954853be08ecf627d6af00)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:26.286486 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598081ff-099c-4aef-bf17-e2e633a9483f None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:26.288621 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 105/207] 10.222.0.55 () {56 vars in 1186 bytes} [Fri Sep 23 14:58:26 2022] GET /identity/v3/projects/0ffbe3ca86954853be08ecf627d6af00 => generated 298 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:58:26.294451 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40a089a-f88c-4561-b737-b2867edfc840 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:26.430476 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40a089a-f88c-4561-b737-b2867edfc840 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:26.441615 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:26.476343 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56940867-b545-4f2f-97e9-bf45e518a502 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:26.476976 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56940867-b545-4f2f-97e9-bf45e518a502 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:26.477261 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56940867-b545-4f2f-97e9-bf45e518a502 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:26.477536 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56940867-b545-4f2f-97e9-bf45e518a502 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:26.479137 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56940867-b545-4f2f-97e9-bf45e518a502 None admin] RBAC: Adding query filter params (role.id=ea002840090d4194bb75fa083a55dc5f, scope.project.id=0ffbe3ca86954853be08ecf627d6af00, user.id=fc9052ae566f4bd2ab726f74f7a156cd) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:58:26.479453 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56940867-b545-4f2f-97e9-bf45e518a502 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:26.481011 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56940867-b545-4f2f-97e9-bf45e518a502 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:26.488563 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 103/208] 10.222.0.55 () {56 vars in 1374 bytes} [Fri Sep 23 14:58:26 2022] GET /identity/v3/role_assignments?user.id=fc9052ae566f4bd2ab726f74f7a156cd&scope.project.id=0ffbe3ca86954853be08ecf627d6af00&role.id=ea002840090d4194bb75fa083a55dc5f => generated 608 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:58:26.740743 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe366be-cc28-4cdd-9f35-501688d75e1e None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:26.741249 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe366be-cc28-4cdd-9f35-501688d75e1e None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:26.741249 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe366be-cc28-4cdd-9f35-501688d75e1e None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:26.742604 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 106/209] 10.222.0.55 () {54 vars in 927 bytes} [Fri Sep 23 14:58:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 14:58:26.749037 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59fc8b1-a584-4155-9ac9-1cd59086db9c None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:26.749331 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59fc8b1-a584-4155-9ac9-1cd59086db9c None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:26.749611 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59fc8b1-a584-4155-9ac9-1cd59086db9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:26.772466 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-f59fc8b1-a584-4155-9ac9-1cd59086db9c None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 14:58:26.773675 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-f59fc8b1-a584-4155-9ac9-1cd59086db9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 14:58:26.875639 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 104/210] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Sep 23 14:58:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 14:58:26.951760 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6911dbe5-9df9-4ccc-ae11-5b3ba9262bd2 None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:26.952122 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6911dbe5-9df9-4ccc-ae11-5b3ba9262bd2 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:26.952363 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6911dbe5-9df9-4ccc-ae11-5b3ba9262bd2 None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:26.954118 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 107/211] 10.222.0.55 () {54 vars in 927 bytes} [Fri Sep 23 14:58:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 14:58:26.959515 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096afaa6-4218-4912-b9ec-560f5d648f5f None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:26.959818 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096afaa6-4218-4912-b9ec-560f5d648f5f None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:26.960115 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096afaa6-4218-4912-b9ec-560f5d648f5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:26.982614 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-096afaa6-4218-4912-b9ec-560f5d648f5f None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 14:58:26.984257 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-096afaa6-4218-4912-b9ec-560f5d648f5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 14:58:27.060183 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 105/212] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Sep 23 14:58:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 14:58:27.236911 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6911dbe5-9df9-4ccc-ae11-5b3ba9262bd2 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:27.299711 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:27.348798 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6911dbe5-9df9-4ccc-ae11-5b3ba9262bd2 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:27.376042 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096afaa6-4218-4912-b9ec-560f5d648f5f None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:27.393471 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1609d6a2-734b-4470-99cf-660e98e4a179 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:27.394415 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1609d6a2-734b-4470-99cf-660e98e4a179 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:27.394703 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1609d6a2-734b-4470-99cf-660e98e4a179 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:27.394980 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1609d6a2-734b-4470-99cf-660e98e4a179 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:27.410143 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1609d6a2-734b-4470-99cf-660e98e4a179 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 14:58:27.412150 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:27.414934 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-1609d6a2-734b-4470-99cf-660e98e4a179 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 23 14:58:27.414934 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 14:58:27.414934 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 14:58:27.414934 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 14:58:27.414934 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 14:58:27.414934 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 14:58:27.414934 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 14:58:27.414934 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 14:58:27.414934 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 14:58:27.414934 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 14:58:27.414934 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 14:58:27.414934 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 14:58:27.414934 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 14:58:27.414934 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 14:58:27.414934 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 14:58:27.414934 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 14:58:27.415876 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 14:58:27.415876 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 14:58:27.415876 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 14:58:27.415876 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 14:58:27.415876 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 14:58:27.415876 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 14:58:27.415876 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 14:58:27.415876 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 14:58:27.415876 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 14:58:27.415876 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 14:58:27.415876 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 14:58:27.415876 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Sep 23 14:58:27.415876 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 14:58:27.415876 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Sep 23 14:58:27.415876 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 14:58:27.415876 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 14:58:27.416725 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 14:58:27.416725 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Sep 23 14:58:27.416725 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 14:58:27.416725 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Sep 23 14:58:27.416725 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 14:58:27.416725 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 14:58:27.416725 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 14:58:27.416725 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 14:58:27.416725 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 14:58:27.416725 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 14:58:27.416725 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 23 14:58:27.416725 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Sep 23 14:58:27.416725 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 108/213] 10.222.0.55 () {56 vars in 1099 bytes} [Fri Sep 23 14:58:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 14:58:27.423684 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1609d6a2-734b-4470-99cf-660e98e4a179 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:27.493691 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096afaa6-4218-4912-b9ec-560f5d648f5f None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:27.531281 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f497c1-7ed2-4add-8084-f5a4be698899 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:27.532618 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f497c1-7ed2-4add-8084-f5a4be698899 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:27.533142 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f497c1-7ed2-4add-8084-f5a4be698899 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:27.533631 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f497c1-7ed2-4add-8084-f5a4be698899 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:27.534944 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f497c1-7ed2-4add-8084-f5a4be698899 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:27.536856 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f497c1-7ed2-4add-8084-f5a4be698899 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:27.539155 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 106/214] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Sep 23 14:58:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:58:27.539270 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1609d6a2-734b-4470-99cf-660e98e4a179 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:27.546331 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f497c1-7ed2-4add-8084-f5a4be698899 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:27.566102 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:27.596337 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d81a2b5-ba5d-4c14-a81b-b9bdfba9e2c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:27.596995 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d81a2b5-ba5d-4c14-a81b-b9bdfba9e2c3 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:27.597295 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d81a2b5-ba5d-4c14-a81b-b9bdfba9e2c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:27.597602 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d81a2b5-ba5d-4c14-a81b-b9bdfba9e2c3 None admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:27.598354 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d81a2b5-ba5d-4c14-a81b-b9bdfba9e2c3 None admin] RBAC: Adding query filter params (name=member) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:58:27.598698 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d81a2b5-ba5d-4c14-a81b-b9bdfba9e2c3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:27.601499 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d81a2b5-ba5d-4c14-a81b-b9bdfba9e2c3 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:27.619259 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 109/215] 10.222.0.55 () {56 vars in 1101 bytes} [Fri Sep 23 14:58:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 14:58:27.620918 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d81a2b5-ba5d-4c14-a81b-b9bdfba9e2c3 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:27.691107 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:27.695742 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f497c1-7ed2-4add-8084-f5a4be698899 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:27.740979 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe57def-b54b-4264-8a3e-76bd33c9407c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:27.742119 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe57def-b54b-4264-8a3e-76bd33c9407c None admin] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:27.742862 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe57def-b54b-4264-8a3e-76bd33c9407c None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:27.744966 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe57def-b54b-4264-8a3e-76bd33c9407c None admin] PATH_INFO: `/v3/users` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:27.745982 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe57def-b54b-4264-8a3e-76bd33c9407c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:27.748383 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe57def-b54b-4264-8a3e-76bd33c9407c None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:27.773893 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d81a2b5-ba5d-4c14-a81b-b9bdfba9e2c3 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:27.814682 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b117d1-39c5-4132-ad20-18f402d3c07b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:27.815321 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b117d1-39c5-4132-ad20-18f402d3c07b None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:27.815607 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b117d1-39c5-4132-ad20-18f402d3c07b None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:27.815884 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b117d1-39c5-4132-ad20-18f402d3c07b None admin] PATH_INFO: `/v3/users/4b5336d82912467b9cf6aa369a36d7e0` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:27.830480 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b117d1-39c5-4132-ad20-18f402d3c07b None admin] RBAC: Authorizing `identity:get_user(user_id=4b5336d82912467b9cf6aa369a36d7e0)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:27.831940 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b117d1-39c5-4132-ad20-18f402d3c07b None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:27.833627 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 110/216] 10.222.0.55 () {56 vars in 1177 bytes} [Fri Sep 23 14:58:27 2022] GET /identity/v3/users/4b5336d82912467b9cf6aa369a36d7e0 => generated 297 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:58:27.838759 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3643dcbc-b498-4b84-884d-6fdadb7d3c00 None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:27.838759 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3643dcbc-b498-4b84-884d-6fdadb7d3c00 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:27.838759 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3643dcbc-b498-4b84-884d-6fdadb7d3c00 None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:27.838759 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 111/217] 10.222.0.55 () {54 vars in 927 bytes} [Fri Sep 23 14:58:27 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 14:58:27.840899 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3643dcbc-b498-4b84-884d-6fdadb7d3c00 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:27.873366 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:27.886880 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 107/218] 10.222.0.55 () {60 vars in 1132 bytes} [Fri Sep 23 14:58:27 2022] POST /identity/v3/users => generated 297 bytes in 341 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 14:58:27.888135 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5eac49-4390-4cd2-bbd2-db05a2f22d07 None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:27.888521 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5eac49-4390-4cd2-bbd2-db05a2f22d07 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:27.888901 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5eac49-4390-4cd2-bbd2-db05a2f22d07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:27.924039 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-4b5eac49-4390-4cd2-bbd2-db05a2f22d07 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 14:58:27.926456 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-4b5eac49-4390-4cd2-bbd2-db05a2f22d07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 14:58:27.975937 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3643dcbc-b498-4b84-884d-6fdadb7d3c00 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:28.017231 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 108/219] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Sep 23 14:58:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 14:58:28.023032 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f027507c-2abd-4546-93de-2287e7712c1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:28.024074 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f027507c-2abd-4546-93de-2287e7712c1b None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:28.025212 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f027507c-2abd-4546-93de-2287e7712c1b None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:28.025777 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f027507c-2abd-4546-93de-2287e7712c1b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:28.101787 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f027507c-2abd-4546-93de-2287e7712c1b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:28.103312 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f027507c-2abd-4546-93de-2287e7712c1b None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:28.134686 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:28.208192 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 112/220] 10.222.0.55 () {58 vars in 1282 bytes} [Fri Sep 23 14:58:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 23 14:58:28.214702 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5eac49-4390-4cd2-bbd2-db05a2f22d07 None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:28.316522 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f027507c-2abd-4546-93de-2287e7712c1b None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:28.331999 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:28.349210 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5eac49-4390-4cd2-bbd2-db05a2f22d07 None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:28.388744 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9841b087-3ce6-4eea-8a00-03da84a904a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:28.389695 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9841b087-3ce6-4eea-8a00-03da84a904a7 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:28.390149 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9841b087-3ce6-4eea-8a00-03da84a904a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:28.390590 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9841b087-3ce6-4eea-8a00-03da84a904a7 None admin] PATH_INFO: `/v3/projects/346756b911774d55af2a9ad3de2143b4` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:28.392084 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9841b087-3ce6-4eea-8a00-03da84a904a7 None admin] RBAC: Authorizing `identity:get_project(project_id=346756b911774d55af2a9ad3de2143b4)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:28.393893 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9841b087-3ce6-4eea-8a00-03da84a904a7 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:28.397410 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 109/221] 10.222.0.55 () {56 vars in 1186 bytes} [Fri Sep 23 14:58:28 2022] GET /identity/v3/projects/346756b911774d55af2a9ad3de2143b4 => generated 302 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:58:28.403721 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9841b087-3ce6-4eea-8a00-03da84a904a7 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:28.445432 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f027507c-2abd-4546-93de-2287e7712c1b None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:28.487345 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:28.494077 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3575df10-616a-4448-9ce0-097f2683c85b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:28.494737 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3575df10-616a-4448-9ce0-097f2683c85b None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:28.495026 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3575df10-616a-4448-9ce0-097f2683c85b None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:28.495308 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3575df10-616a-4448-9ce0-097f2683c85b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:28.515495 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3575df10-616a-4448-9ce0-097f2683c85b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 14:58:28.525130 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-3575df10-616a-4448-9ce0-097f2683c85b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 23 14:58:28.525130 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 14:58:28.525130 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 14:58:28.525130 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 14:58:28.525130 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 14:58:28.525130 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 14:58:28.525130 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 14:58:28.525130 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 14:58:28.525130 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 14:58:28.525130 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 14:58:28.525130 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 14:58:28.525130 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 14:58:28.525130 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 14:58:28.525130 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 14:58:28.525130 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 14:58:28.525130 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 14:58:28.526177 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 14:58:28.526177 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 14:58:28.526177 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 14:58:28.526177 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 14:58:28.526177 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 14:58:28.526177 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 14:58:28.526177 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 14:58:28.526177 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 14:58:28.526177 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 14:58:28.526177 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 14:58:28.526177 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 14:58:28.526177 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Sep 23 14:58:28.526177 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 14:58:28.526177 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Sep 23 14:58:28.526177 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 14:58:28.526177 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 14:58:28.526950 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 14:58:28.526950 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Sep 23 14:58:28.526950 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 14:58:28.526950 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Sep 23 14:58:28.526950 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 14:58:28.526950 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 14:58:28.526950 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 14:58:28.526950 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 14:58:28.526950 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 14:58:28.526950 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 14:58:28.526950 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 23 14:58:28.526950 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Sep 23 14:58:28.526950 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 113/222] 10.222.0.55 () {56 vars in 1096 bytes} [Fri Sep 23 14:58:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 14:58:28.536267 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3575df10-616a-4448-9ce0-097f2683c85b None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:28.541453 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9841b087-3ce6-4eea-8a00-03da84a904a7 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:28.580102 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524d271a-a34b-4c17-b6f7-86e3f4281a0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:28.580860 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524d271a-a34b-4c17-b6f7-86e3f4281a0b None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:28.581213 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524d271a-a34b-4c17-b6f7-86e3f4281a0b None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:28.581500 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524d271a-a34b-4c17-b6f7-86e3f4281a0b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:28.582236 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524d271a-a34b-4c17-b6f7-86e3f4281a0b None admin] RBAC: Adding query filter params (role.id=af257be2dee54d6b942f58650196582c, scope.project.id=346756b911774d55af2a9ad3de2143b4, user.id=4b5336d82912467b9cf6aa369a36d7e0) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:58:28.582545 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524d271a-a34b-4c17-b6f7-86e3f4281a0b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:28.584093 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524d271a-a34b-4c17-b6f7-86e3f4281a0b None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:28.591649 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 110/223] 10.222.0.55 () {56 vars in 1374 bytes} [Fri Sep 23 14:58:28 2022] GET /identity/v3/role_assignments?user.id=4b5336d82912467b9cf6aa369a36d7e0&scope.project.id=346756b911774d55af2a9ad3de2143b4&role.id=af257be2dee54d6b942f58650196582c => generated 259 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:58:28.699398 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3575df10-616a-4448-9ce0-097f2683c85b None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:28.740918 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f7d885-573c-4605-bc43-ef5aebb7c4b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:28.741607 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f7d885-573c-4605-bc43-ef5aebb7c4b3 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:28.741922 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f7d885-573c-4605-bc43-ef5aebb7c4b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:28.742236 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f7d885-573c-4605-bc43-ef5aebb7c4b3 None admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:28.742917 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f7d885-573c-4605-bc43-ef5aebb7c4b3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:58:28.743906 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f7d885-573c-4605-bc43-ef5aebb7c4b3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:28.745483 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f7d885-573c-4605-bc43-ef5aebb7c4b3 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:28.755719 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 114/224] 10.222.0.55 () {56 vars in 1099 bytes} [Fri Sep 23 14:58:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:58:28.761659 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524d271a-a34b-4c17-b6f7-86e3f4281a0b None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:28.783296 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:28.826563 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da351bf7-9545-43ce-9c60-32b5f3cfe262 None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:28.827073 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da351bf7-9545-43ce-9c60-32b5f3cfe262 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:28.827160 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da351bf7-9545-43ce-9c60-32b5f3cfe262 None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:28.828460 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 115/225] 10.222.0.55 () {54 vars in 927 bytes} [Fri Sep 23 14:58:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 14:58:28.833922 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baefcf6a-82cd-4238-bf07-d2b63d0d120a None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:28.834207 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baefcf6a-82cd-4238-bf07-d2b63d0d120a None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:28.834480 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baefcf6a-82cd-4238-bf07-d2b63d0d120a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:28.855182 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-baefcf6a-82cd-4238-bf07-d2b63d0d120a None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 14:58:28.856543 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-baefcf6a-82cd-4238-bf07-d2b63d0d120a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 14:58:28.910750 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:28.937583 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 116/226] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Sep 23 14:58:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 23 14:58:28.942908 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524d271a-a34b-4c17-b6f7-86e3f4281a0b None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:28.994718 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acd8348-467f-4956-8fff-6020247901d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:28.995393 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acd8348-467f-4956-8fff-6020247901d9 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:28.995678 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acd8348-467f-4956-8fff-6020247901d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:28.995952 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acd8348-467f-4956-8fff-6020247901d9 None admin] PATH_INFO: `/v3/users/755ef177078341b6a1f30e126cb4a9fa` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:28.997156 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acd8348-467f-4956-8fff-6020247901d9 None admin] RBAC: Authorizing `identity:get_user(user_id=755ef177078341b6a1f30e126cb4a9fa)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:28.998682 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acd8348-467f-4956-8fff-6020247901d9 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:29.000740 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 111/227] 10.222.0.55 () {56 vars in 1177 bytes} [Fri Sep 23 14:58:28 2022] GET /identity/v3/users/755ef177078341b6a1f30e126cb4a9fa => generated 283 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:58:29.009180 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baefcf6a-82cd-4238-bf07-d2b63d0d120a None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:29.164664 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baefcf6a-82cd-4238-bf07-d2b63d0d120a None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:29.219701 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf06dd2-c2a2-4c1e-b061-64b1c626a21f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:29.220394 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf06dd2-c2a2-4c1e-b061-64b1c626a21f None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:29.220672 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf06dd2-c2a2-4c1e-b061-64b1c626a21f None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:29.220944 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf06dd2-c2a2-4c1e-b061-64b1c626a21f None admin] PATH_INFO: `/v3/projects/346756b911774d55af2a9ad3de2143b4` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:29.222087 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf06dd2-c2a2-4c1e-b061-64b1c626a21f None admin] RBAC: Authorizing `identity:get_project(project_id=346756b911774d55af2a9ad3de2143b4)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:29.223527 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf06dd2-c2a2-4c1e-b061-64b1c626a21f None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:29.225557 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 117/228] 10.222.0.55 () {56 vars in 1186 bytes} [Fri Sep 23 14:58:29 2022] GET /identity/v3/projects/346756b911774d55af2a9ad3de2143b4 => generated 302 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:58:29.232176 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acd8348-467f-4956-8fff-6020247901d9 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:29.277826 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf06dd2-c2a2-4c1e-b061-64b1c626a21f None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:29.349792 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:29.381960 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:29.418115 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acd8348-467f-4956-8fff-6020247901d9 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:29.463009 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf06dd2-c2a2-4c1e-b061-64b1c626a21f None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:29.471261 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f37e3d-6945-4f0f-a878-634adaa9bbde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:29.471966 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f37e3d-6945-4f0f-a878-634adaa9bbde None admin] REQUEST_METHOD: `PUT` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:29.472281 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f37e3d-6945-4f0f-a878-634adaa9bbde None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:29.472617 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f37e3d-6945-4f0f-a878-634adaa9bbde None admin] PATH_INFO: `/v3/projects/346756b911774d55af2a9ad3de2143b4/users/755ef177078341b6a1f30e126cb4a9fa/roles/8e2919e79df040e8926f21607818b58c` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:29.474704 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f37e3d-6945-4f0f-a878-634adaa9bbde None admin] RBAC: Authorizing `identity:create_grant(project_id=346756b911774d55af2a9ad3de2143b4, user_id=755ef177078341b6a1f30e126cb4a9fa, role_id=8e2919e79df040e8926f21607818b58c)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:29.476214 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f37e3d-6945-4f0f-a878-634adaa9bbde None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:29.516216 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0eecea-23ae-46ce-9397-08643ac964f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:29.516914 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0eecea-23ae-46ce-9397-08643ac964f2 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:29.517241 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0eecea-23ae-46ce-9397-08643ac964f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:29.517584 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0eecea-23ae-46ce-9397-08643ac964f2 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:29.530412 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0eecea-23ae-46ce-9397-08643ac964f2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 14:58:29.535833 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-7a0eecea-23ae-46ce-9397-08643ac964f2 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 23 14:58:29.535833 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 14:58:29.535833 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 14:58:29.535833 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 14:58:29.535833 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 14:58:29.535833 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 14:58:29.535833 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 14:58:29.535833 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 14:58:29.535833 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 14:58:29.535833 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 14:58:29.535833 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 14:58:29.535833 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 14:58:29.535833 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 14:58:29.535833 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 14:58:29.535833 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 14:58:29.535833 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 14:58:29.536738 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 14:58:29.536738 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 14:58:29.536738 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 14:58:29.536738 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 14:58:29.536738 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 14:58:29.536738 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 14:58:29.536738 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 14:58:29.536738 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 14:58:29.536738 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 14:58:29.536738 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 14:58:29.536738 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 14:58:29.536738 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Sep 23 14:58:29.536738 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 14:58:29.536738 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Sep 23 14:58:29.536738 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 14:58:29.536738 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 14:58:29.538001 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 14:58:29.538001 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Sep 23 14:58:29.538001 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 14:58:29.538001 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Sep 23 14:58:29.538001 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 14:58:29.538001 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 14:58:29.538001 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 14:58:29.538001 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 14:58:29.538001 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 14:58:29.538001 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 14:58:29.538001 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 23 14:58:29.538001 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Sep 23 14:58:29.538001 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 118/229] 10.222.0.55 () {56 vars in 1114 bytes} [Fri Sep 23 14:58:29 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 14:58:29.551328 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 112/230] 10.222.0.55 () {58 vars in 1439 bytes} [Fri Sep 23 14:58:29 2022] PUT /identity/v3/projects/346756b911774d55af2a9ad3de2143b4/users/755ef177078341b6a1f30e126cb4a9fa/roles/8e2919e79df040e8926f21607818b58c => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 14:58:29.551663 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0eecea-23ae-46ce-9397-08643ac964f2 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:29.697461 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0eecea-23ae-46ce-9397-08643ac964f2 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:29.717135 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:29.743286 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fac626-e78f-440f-8a33-8cadb52ffc40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:29.744087 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fac626-e78f-440f-8a33-8cadb52ffc40 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:29.744291 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fac626-e78f-440f-8a33-8cadb52ffc40 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:29.744526 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fac626-e78f-440f-8a33-8cadb52ffc40 None admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:29.745220 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fac626-e78f-440f-8a33-8cadb52ffc40 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:58:29.745526 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fac626-e78f-440f-8a33-8cadb52ffc40 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:29.749404 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fac626-e78f-440f-8a33-8cadb52ffc40 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:29.760610 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 119/231] 10.222.0.55 () {56 vars in 1111 bytes} [Fri Sep 23 14:58:29 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:58:29.768367 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f37e3d-6945-4f0f-a878-634adaa9bbde None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:29.912641 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f37e3d-6945-4f0f-a878-634adaa9bbde None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:29.959934 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31072242-fe51-429e-8aaf-8947b48a5c3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:29.960610 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31072242-fe51-429e-8aaf-8947b48a5c3b None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:29.960899 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31072242-fe51-429e-8aaf-8947b48a5c3b None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:29.961170 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31072242-fe51-429e-8aaf-8947b48a5c3b None admin] PATH_INFO: `/v3/users/755ef177078341b6a1f30e126cb4a9fa` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:29.962352 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31072242-fe51-429e-8aaf-8947b48a5c3b None admin] RBAC: Authorizing `identity:get_user(user_id=755ef177078341b6a1f30e126cb4a9fa)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:29.963825 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31072242-fe51-429e-8aaf-8947b48a5c3b None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:29.965572 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 113/232] 10.222.0.55 () {56 vars in 1177 bytes} [Fri Sep 23 14:58:29 2022] GET /identity/v3/users/755ef177078341b6a1f30e126cb4a9fa => generated 283 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:58:29.971572 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fac626-e78f-440f-8a33-8cadb52ffc40 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:30.112945 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fac626-e78f-440f-8a33-8cadb52ffc40 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:30.148543 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7038e398-08f7-4e69-88c6-51e089ce6f87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:30.149188 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7038e398-08f7-4e69-88c6-51e089ce6f87 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:30.149520 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7038e398-08f7-4e69-88c6-51e089ce6f87 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:30.149795 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7038e398-08f7-4e69-88c6-51e089ce6f87 None admin] PATH_INFO: `/v3/projects/346756b911774d55af2a9ad3de2143b4` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:30.150933 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7038e398-08f7-4e69-88c6-51e089ce6f87 None admin] RBAC: Authorizing `identity:get_project(project_id=346756b911774d55af2a9ad3de2143b4)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:30.152409 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7038e398-08f7-4e69-88c6-51e089ce6f87 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:30.154230 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 120/233] 10.222.0.55 () {56 vars in 1186 bytes} [Fri Sep 23 14:58:29 2022] GET /identity/v3/projects/346756b911774d55af2a9ad3de2143b4 => generated 302 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:58:30.161617 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31072242-fe51-429e-8aaf-8947b48a5c3b None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:30.191315 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:30.323067 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31072242-fe51-429e-8aaf-8947b48a5c3b None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:30.371577 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe91114a-5cbf-49b4-88d6-f88cffa1e03d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:30.372278 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe91114a-5cbf-49b4-88d6-f88cffa1e03d None admin] REQUEST_METHOD: `PUT` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:30.372557 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe91114a-5cbf-49b4-88d6-f88cffa1e03d None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:30.372827 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe91114a-5cbf-49b4-88d6-f88cffa1e03d None admin] PATH_INFO: `/v3/projects/346756b911774d55af2a9ad3de2143b4/users/755ef177078341b6a1f30e126cb4a9fa/roles/5c74fdc0e9cc44538cd8282006ed4443` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:30.375049 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe91114a-5cbf-49b4-88d6-f88cffa1e03d None admin] RBAC: Authorizing `identity:create_grant(project_id=346756b911774d55af2a9ad3de2143b4, user_id=755ef177078341b6a1f30e126cb4a9fa, role_id=5c74fdc0e9cc44538cd8282006ed4443)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:30.376551 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe91114a-5cbf-49b4-88d6-f88cffa1e03d None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:30.482465 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 114/234] 10.222.0.55 () {58 vars in 1439 bytes} [Fri Sep 23 14:58:30 2022] PUT /identity/v3/projects/346756b911774d55af2a9ad3de2143b4/users/755ef177078341b6a1f30e126cb4a9fa/roles/5c74fdc0e9cc44538cd8282006ed4443 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 14:58:30.885479 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2374ff-0399-4635-b01c-75e5e2ad66e2 None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:30.885657 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2374ff-0399-4635-b01c-75e5e2ad66e2 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:30.885936 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2374ff-0399-4635-b01c-75e5e2ad66e2 None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:30.887520 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 121/235] 10.222.0.55 () {54 vars in 927 bytes} [Fri Sep 23 14:58:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 14:58:30.893040 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd9848e-65e6-4786-8d02-f85d23e42ad8 None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:30.893601 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd9848e-65e6-4786-8d02-f85d23e42ad8 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:30.894065 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd9848e-65e6-4786-8d02-f85d23e42ad8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:30.916162 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-fbd9848e-65e6-4786-8d02-f85d23e42ad8 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 14:58:30.917665 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-fbd9848e-65e6-4786-8d02-f85d23e42ad8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 14:58:30.968955 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c161b2e1-9ff6-4981-9d0b-538e1435b1c0 None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:30.969611 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c161b2e1-9ff6-4981-9d0b-538e1435b1c0 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:30.970089 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c161b2e1-9ff6-4981-9d0b-538e1435b1c0 None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:30.972787 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 122/236] 10.222.0.55 () {54 vars in 927 bytes} [Fri Sep 23 14:58:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 14:58:30.978294 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84284f84-fdd7-485f-9c2c-062a461ab7ce None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:30.978948 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84284f84-fdd7-485f-9c2c-062a461ab7ce None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:30.979643 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84284f84-fdd7-485f-9c2c-062a461ab7ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:31.011736 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-84284f84-fdd7-485f-9c2c-062a461ab7ce None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 14:58:31.014107 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-84284f84-fdd7-485f-9c2c-062a461ab7ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 14:58:31.026345 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 115/237] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Sep 23 14:58:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 14:58:31.059352 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:31.112036 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 123/238] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Sep 23 14:58:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 14:58:31.335618 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd9848e-65e6-4786-8d02-f85d23e42ad8 None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:31.340142 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:31.350715 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e252247-36d8-45d7-bdc8-8180177db278 None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:31.350995 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e252247-36d8-45d7-bdc8-8180177db278 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:31.355866 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e252247-36d8-45d7-bdc8-8180177db278 None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:31.355866 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 124/239] 10.222.0.55 () {54 vars in 927 bytes} [Fri Sep 23 14:58:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 14:58:31.359844 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3d8c52-1c33-4a12-9448-8f0f57cf8e67 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:31.360287 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3d8c52-1c33-4a12-9448-8f0f57cf8e67 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:31.360668 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3d8c52-1c33-4a12-9448-8f0f57cf8e67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:31.381279 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-1a3d8c52-1c33-4a12-9448-8f0f57cf8e67 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 14:58:31.382619 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-1a3d8c52-1c33-4a12-9448-8f0f57cf8e67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 14:58:31.487713 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 125/240] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Sep 23 14:58:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 23 14:58:31.506830 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd9848e-65e6-4786-8d02-f85d23e42ad8 None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:31.550920 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6d745a-0e34-4c5f-aecc-df2571c9fa01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:31.551574 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6d745a-0e34-4c5f-aecc-df2571c9fa01 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:31.551856 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6d745a-0e34-4c5f-aecc-df2571c9fa01 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:31.552160 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6d745a-0e34-4c5f-aecc-df2571c9fa01 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:31.554136 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3d8c52-1c33-4a12-9448-8f0f57cf8e67 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:31.581548 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6d745a-0e34-4c5f-aecc-df2571c9fa01 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 14:58:31.593804 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application [None req-ab6d745a-0e34-4c5f-aecc-df2571c9fa01 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 23 14:58:31.593804 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 14:58:31.593804 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 14:58:31.593804 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 14:58:31.593804 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 14:58:31.593804 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 14:58:31.593804 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 14:58:31.593804 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 14:58:31.593804 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 14:58:31.593804 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 14:58:31.593804 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 14:58:31.593804 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 14:58:31.593804 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 14:58:31.593804 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 14:58:31.593804 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 14:58:31.593804 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 14:58:31.594676 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 14:58:31.594676 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 14:58:31.594676 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 14:58:31.594676 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 14:58:31.594676 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 14:58:31.594676 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 14:58:31.594676 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 14:58:31.594676 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 14:58:31.594676 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 14:58:31.594676 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 14:58:31.594676 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 14:58:31.594676 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application with Lock( Sep 23 14:58:31.594676 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 14:58:31.594676 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._enter() Sep 23 14:58:31.594676 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 14:58:31.594676 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 14:58:31.595461 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 14:58:31.595461 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.creator() Sep 23 14:58:31.595461 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 14:58:31.595461 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application created_value = creator( Sep 23 14:58:31.595461 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 14:58:31.595461 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 14:58:31.595461 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 14:58:31.595461 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 14:58:31.595461 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 14:58:31.595461 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 14:58:31.595461 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 23 14:58:31.595461 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Sep 23 14:58:31.596602 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 116/241] 10.222.0.55 () {56 vars in 1099 bytes} [Fri Sep 23 14:58:31 2022] GET /identity/v3/roles/reader => generated 84 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 14:58:31.602255 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6d745a-0e34-4c5f-aecc-df2571c9fa01 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:31.699074 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3d8c52-1c33-4a12-9448-8f0f57cf8e67 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:31.700774 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:31.738761 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849f5572-e195-45bc-8383-da3bd680205c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:31.739445 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849f5572-e195-45bc-8383-da3bd680205c None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:31.739740 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849f5572-e195-45bc-8383-da3bd680205c None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:31.740052 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849f5572-e195-45bc-8383-da3bd680205c None admin] PATH_INFO: `/v3/domains/default` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:31.741179 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849f5572-e195-45bc-8383-da3bd680205c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:31.742623 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849f5572-e195-45bc-8383-da3bd680205c None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:31.745329 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 126/242] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Sep 23 14:58:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 14:58:31.750792 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849f5572-e195-45bc-8383-da3bd680205c None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:31.755499 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:31.836715 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6d745a-0e34-4c5f-aecc-df2571c9fa01 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:31.875164 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18af4f3a-11c6-4a52-a20f-945abf02f4d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:31.876265 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18af4f3a-11c6-4a52-a20f-945abf02f4d8 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:31.876742 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18af4f3a-11c6-4a52-a20f-945abf02f4d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:31.877206 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18af4f3a-11c6-4a52-a20f-945abf02f4d8 None admin] PATH_INFO: `/v3/roles` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:31.878061 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18af4f3a-11c6-4a52-a20f-945abf02f4d8 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:58:31.878577 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18af4f3a-11c6-4a52-a20f-945abf02f4d8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:31.880352 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18af4f3a-11c6-4a52-a20f-945abf02f4d8 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:31.890541 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 117/243] 10.222.0.55 () {56 vars in 1101 bytes} [Fri Sep 23 14:58:31 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 14:58:31.896432 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18af4f3a-11c6-4a52-a20f-945abf02f4d8 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:31.898945 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849f5572-e195-45bc-8383-da3bd680205c None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:31.937899 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a469cbe-2710-4523-a9ab-2dbaf05eabcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:31.938518 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a469cbe-2710-4523-a9ab-2dbaf05eabcb None admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:31.938804 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a469cbe-2710-4523-a9ab-2dbaf05eabcb None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:31.939083 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a469cbe-2710-4523-a9ab-2dbaf05eabcb None admin] PATH_INFO: `/v3/users` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:31.939739 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a469cbe-2710-4523-a9ab-2dbaf05eabcb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:31.941219 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a469cbe-2710-4523-a9ab-2dbaf05eabcb None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:32.006669 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18af4f3a-11c6-4a52-a20f-945abf02f4d8 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:32.034919 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 127/244] 10.222.0.55 () {60 vars in 1132 bytes} [Fri Sep 23 14:58:31 2022] POST /identity/v3/users => generated 293 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 14:58:32.036992 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a469cbe-2710-4523-a9ab-2dbaf05eabcb None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:32.042648 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:32.061526 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebc30a2-e727-499b-bf6c-6c3cd00a5187 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:32.062151 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebc30a2-e727-499b-bf6c-6c3cd00a5187 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:32.062436 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebc30a2-e727-499b-bf6c-6c3cd00a5187 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:32.062705 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebc30a2-e727-499b-bf6c-6c3cd00a5187 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:32.078475 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:32.093116 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebc30a2-e727-499b-bf6c-6c3cd00a5187 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 14:58:32.097892 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application [None req-9ebc30a2-e727-499b-bf6c-6c3cd00a5187 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 23 14:58:32.097892 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 14:58:32.097892 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 14:58:32.097892 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 14:58:32.097892 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 14:58:32.097892 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 14:58:32.097892 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 14:58:32.097892 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 14:58:32.097892 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 14:58:32.097892 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 14:58:32.097892 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 14:58:32.097892 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 14:58:32.097892 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 14:58:32.097892 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 14:58:32.097892 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 14:58:32.097892 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 14:58:32.098771 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 14:58:32.098771 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 14:58:32.098771 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 14:58:32.098771 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 14:58:32.098771 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 14:58:32.098771 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 14:58:32.098771 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 14:58:32.098771 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 14:58:32.098771 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 14:58:32.098771 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 14:58:32.098771 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 14:58:32.098771 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application with Lock( Sep 23 14:58:32.098771 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 14:58:32.098771 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._enter() Sep 23 14:58:32.098771 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 14:58:32.098771 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 14:58:32.099671 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 14:58:32.099671 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.creator() Sep 23 14:58:32.099671 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 14:58:32.099671 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application created_value = creator( Sep 23 14:58:32.099671 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 14:58:32.099671 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 14:58:32.099671 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 14:58:32.099671 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 14:58:32.099671 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 14:58:32.099671 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 14:58:32.099671 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 23 14:58:32.099671 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Sep 23 14:58:32.104516 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 118/245] 10.222.0.55 () {56 vars in 1099 bytes} [Fri Sep 23 14:58:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 14:58:32.110241 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebc30a2-e727-499b-bf6c-6c3cd00a5187 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:32.182407 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a469cbe-2710-4523-a9ab-2dbaf05eabcb None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:32.227484 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e728cb7-b0e4-487f-914a-71823f20936f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:32.229506 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e728cb7-b0e4-487f-914a-71823f20936f None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:32.230037 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e728cb7-b0e4-487f-914a-71823f20936f None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:32.230502 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e728cb7-b0e4-487f-914a-71823f20936f None admin] PATH_INFO: `/v3/users/3a099085ae5b4b5780b0f3c4d86b19f0` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:32.251076 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e728cb7-b0e4-487f-914a-71823f20936f None admin] RBAC: Authorizing `identity:get_user(user_id=3a099085ae5b4b5780b0f3c4d86b19f0)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:32.253209 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e728cb7-b0e4-487f-914a-71823f20936f None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:32.256173 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 128/246] 10.222.0.55 () {56 vars in 1177 bytes} [Fri Sep 23 14:58:32 2022] GET /identity/v3/users/3a099085ae5b4b5780b0f3c4d86b19f0 => generated 297 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:58:32.264469 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e728cb7-b0e4-487f-914a-71823f20936f None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:32.376748 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:32.381001 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebc30a2-e727-499b-bf6c-6c3cd00a5187 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:32.399914 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e728cb7-b0e4-487f-914a-71823f20936f None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:32.423190 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765f69b2-ed1d-4cb2-abc6-ab7291615b91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:32.424137 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765f69b2-ed1d-4cb2-abc6-ab7291615b91 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:32.425879 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765f69b2-ed1d-4cb2-abc6-ab7291615b91 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:32.426301 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765f69b2-ed1d-4cb2-abc6-ab7291615b91 None admin] PATH_INFO: `/v3/roles` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:32.427400 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765f69b2-ed1d-4cb2-abc6-ab7291615b91 None admin] RBAC: Adding query filter params (name=member) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:58:32.428127 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765f69b2-ed1d-4cb2-abc6-ab7291615b91 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:32.430421 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765f69b2-ed1d-4cb2-abc6-ab7291615b91 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:32.442427 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 119/247] 10.222.0.55 () {56 vars in 1101 bytes} [Fri Sep 23 14:58:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:58:32.443890 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc53ab28-dc62-415a-aae2-db25a6d8d7ac None None] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:32.444351 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc53ab28-dc62-415a-aae2-db25a6d8d7ac None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:32.444760 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc53ab28-dc62-415a-aae2-db25a6d8d7ac None None] PATH_INFO: `/` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:32.445145 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefa1977-f642-4831-b1f0-4c811f018a7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:32.445772 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefa1977-f642-4831-b1f0-4c811f018a7b None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:32.446498 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 120/248] 10.222.0.55 () {54 vars in 927 bytes} [Fri Sep 23 14:58:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 14:58:32.446618 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefa1977-f642-4831-b1f0-4c811f018a7b None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:32.446918 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefa1977-f642-4831-b1f0-4c811f018a7b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:32.452407 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b799905-4fd4-4b70-8377-482f10a25f5e None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:32.452690 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b799905-4fd4-4b70-8377-482f10a25f5e None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:32.452968 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b799905-4fd4-4b70-8377-482f10a25f5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:32.475163 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-8b799905-4fd4-4b70-8377-482f10a25f5e None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 14:58:32.476487 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-8b799905-4fd4-4b70-8377-482f10a25f5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 14:58:32.539172 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefa1977-f642-4831-b1f0-4c811f018a7b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:32.540940 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:32.541650 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefa1977-f642-4831-b1f0-4c811f018a7b None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:32.570228 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 121/249] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Sep 23 14:58:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 23 14:58:32.571756 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b799905-4fd4-4b70-8377-482f10a25f5e None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:32.648000 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:32.667965 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 129/250] 10.222.0.55 () {58 vars in 1282 bytes} [Fri Sep 23 14:58:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 23 14:58:32.676252 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefa1977-f642-4831-b1f0-4c811f018a7b None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:32.789282 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b799905-4fd4-4b70-8377-482f10a25f5e None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:32.876613 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefa1977-f642-4831-b1f0-4c811f018a7b None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:32.885301 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4941cd-204e-4421-9d9e-fb7461be909c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:32.885951 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4941cd-204e-4421-9d9e-fb7461be909c None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:32.886228 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4941cd-204e-4421-9d9e-fb7461be909c None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:32.886500 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4941cd-204e-4421-9d9e-fb7461be909c None admin] PATH_INFO: `/v3/users/4b5336d82912467b9cf6aa369a36d7e0` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:32.890544 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4941cd-204e-4421-9d9e-fb7461be909c None admin] RBAC: Authorizing `identity:get_user(user_id=4b5336d82912467b9cf6aa369a36d7e0)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:32.892017 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4941cd-204e-4421-9d9e-fb7461be909c None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:32.894094 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 122/251] 10.222.0.55 () {56 vars in 1177 bytes} [Fri Sep 23 14:58:32 2022] GET /identity/v3/users/4b5336d82912467b9cf6aa369a36d7e0 => generated 297 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:58:32.896626 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4941cd-204e-4421-9d9e-fb7461be909c None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:32.952102 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a13a94-f689-4be4-b9a4-48eee6a6d317 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:32.952745 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a13a94-f689-4be4-b9a4-48eee6a6d317 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:32.953022 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a13a94-f689-4be4-b9a4-48eee6a6d317 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:32.953293 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a13a94-f689-4be4-b9a4-48eee6a6d317 None admin] PATH_INFO: `/v3/projects/346756b911774d55af2a9ad3de2143b4` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:32.954571 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a13a94-f689-4be4-b9a4-48eee6a6d317 None admin] RBAC: Authorizing `identity:get_project(project_id=346756b911774d55af2a9ad3de2143b4)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:32.958278 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a13a94-f689-4be4-b9a4-48eee6a6d317 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:32.960165 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 130/252] 10.222.0.55 () {56 vars in 1186 bytes} [Fri Sep 23 14:58:32 2022] GET /identity/v3/projects/346756b911774d55af2a9ad3de2143b4 => generated 302 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 14:58:32.960774 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a13a94-f689-4be4-b9a4-48eee6a6d317 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:32.999011 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:33.072049 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:33.098932 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4941cd-204e-4421-9d9e-fb7461be909c None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:33.167335 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5986eaeb-eeff-4e1a-90c8-aad9f842d4d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:33.168056 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5986eaeb-eeff-4e1a-90c8-aad9f842d4d2 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:33.168337 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5986eaeb-eeff-4e1a-90c8-aad9f842d4d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:33.168633 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5986eaeb-eeff-4e1a-90c8-aad9f842d4d2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:33.170259 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a13a94-f689-4be4-b9a4-48eee6a6d317 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:33.192477 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5986eaeb-eeff-4e1a-90c8-aad9f842d4d2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 14:58:33.199062 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application [None req-5986eaeb-eeff-4e1a-90c8-aad9f842d4d2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 23 14:58:33.199062 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 14:58:33.199062 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 14:58:33.199062 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 14:58:33.199062 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 14:58:33.199062 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 14:58:33.199062 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 14:58:33.199062 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 14:58:33.199062 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 14:58:33.199062 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 14:58:33.199062 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 14:58:33.199062 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 14:58:33.199062 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 14:58:33.199062 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 14:58:33.199062 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 14:58:33.199062 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 14:58:33.200062 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 14:58:33.200062 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 14:58:33.200062 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 14:58:33.200062 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 14:58:33.200062 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 14:58:33.200062 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 14:58:33.200062 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 14:58:33.200062 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 14:58:33.200062 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 14:58:33.200062 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 14:58:33.200062 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 14:58:33.200062 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application with Lock( Sep 23 14:58:33.200062 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 14:58:33.200062 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._enter() Sep 23 14:58:33.200062 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 14:58:33.200062 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 14:58:33.200923 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 14:58:33.200923 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.creator() Sep 23 14:58:33.200923 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 14:58:33.200923 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application created_value = creator( Sep 23 14:58:33.200923 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 14:58:33.200923 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 14:58:33.200923 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 14:58:33.200923 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 14:58:33.200923 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 14:58:33.200923 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 14:58:33.200923 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 23 14:58:33.200923 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Sep 23 14:58:33.201514 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 123/253] 10.222.0.55 () {56 vars in 1096 bytes} [Fri Sep 23 14:58:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 14:58:33.203205 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5986eaeb-eeff-4e1a-90c8-aad9f842d4d2 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:33.238912 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58aa2e33-756d-48b5-b8c3-c876c3fd09b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:33.239709 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58aa2e33-756d-48b5-b8c3-c876c3fd09b1 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:33.240001 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58aa2e33-756d-48b5-b8c3-c876c3fd09b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:33.240285 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58aa2e33-756d-48b5-b8c3-c876c3fd09b1 None admin] PATH_INFO: `/v3/projects/346756b911774d55af2a9ad3de2143b4` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:33.241708 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58aa2e33-756d-48b5-b8c3-c876c3fd09b1 None admin] RBAC: Authorizing `identity:get_project(project_id=346756b911774d55af2a9ad3de2143b4)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:33.246526 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58aa2e33-756d-48b5-b8c3-c876c3fd09b1 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:33.248499 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 131/254] 10.222.0.55 () {56 vars in 1186 bytes} [Fri Sep 23 14:58:32 2022] GET /identity/v3/projects/346756b911774d55af2a9ad3de2143b4 => generated 302 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:58:33.249104 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58aa2e33-756d-48b5-b8c3-c876c3fd09b1 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:33.356481 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5986eaeb-eeff-4e1a-90c8-aad9f842d4d2 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:33.365757 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:33.373394 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58aa2e33-756d-48b5-b8c3-c876c3fd09b1 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:33.395725 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:33.407941 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b7b268-05ca-4964-a2f1-3be8c85f0ae2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:33.408679 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b7b268-05ca-4964-a2f1-3be8c85f0ae2 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:33.409029 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b7b268-05ca-4964-a2f1-3be8c85f0ae2 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:33.409369 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b7b268-05ca-4964-a2f1-3be8c85f0ae2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:33.410202 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b7b268-05ca-4964-a2f1-3be8c85f0ae2 None admin] RBAC: Adding query filter params (role.id=ea002840090d4194bb75fa083a55dc5f, scope.project.id=346756b911774d55af2a9ad3de2143b4, user.id=3a099085ae5b4b5780b0f3c4d86b19f0) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:58:33.410595 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b7b268-05ca-4964-a2f1-3be8c85f0ae2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:33.412310 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d67bd9-62d5-4d94-9b9a-ab7e1d5ea4d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:33.412638 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b7b268-05ca-4964-a2f1-3be8c85f0ae2 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:33.413521 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d67bd9-62d5-4d94-9b9a-ab7e1d5ea4d3 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:33.413950 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d67bd9-62d5-4d94-9b9a-ab7e1d5ea4d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:33.414405 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d67bd9-62d5-4d94-9b9a-ab7e1d5ea4d3 None admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:33.416343 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d67bd9-62d5-4d94-9b9a-ab7e1d5ea4d3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:58:33.416802 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d67bd9-62d5-4d94-9b9a-ab7e1d5ea4d3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:33.418448 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d67bd9-62d5-4d94-9b9a-ab7e1d5ea4d3 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:33.428024 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 124/255] 10.222.0.55 () {56 vars in 1374 bytes} [Fri Sep 23 14:58:33 2022] GET /identity/v3/role_assignments?user.id=3a099085ae5b4b5780b0f3c4d86b19f0&scope.project.id=346756b911774d55af2a9ad3de2143b4&role.id=ea002840090d4194bb75fa083a55dc5f => generated 259 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:58:33.428024 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b7b268-05ca-4964-a2f1-3be8c85f0ae2 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:33.441297 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 132/256] 10.222.0.55 () {56 vars in 1099 bytes} [Fri Sep 23 14:58:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:58:33.442934 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97d6189-eda4-4b78-bac4-24cf7876f85d None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:33.443401 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97d6189-eda4-4b78-bac4-24cf7876f85d None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:33.443918 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97d6189-eda4-4b78-bac4-24cf7876f85d None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:33.449601 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 133/257] 10.222.0.55 () {54 vars in 927 bytes} [Fri Sep 23 14:58:33 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 14:58:33.450876 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97d6189-eda4-4b78-bac4-24cf7876f85d None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:33.552382 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b7b268-05ca-4964-a2f1-3be8c85f0ae2 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:33.598456 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225ea91d-eecf-46f5-81d2-28b26404389c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:33.599137 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225ea91d-eecf-46f5-81d2-28b26404389c None admin] REQUEST_METHOD: `PUT` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:33.599416 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225ea91d-eecf-46f5-81d2-28b26404389c None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:33.599689 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225ea91d-eecf-46f5-81d2-28b26404389c None admin] PATH_INFO: `/v3/projects/346756b911774d55af2a9ad3de2143b4/users/4b5336d82912467b9cf6aa369a36d7e0/roles/af257be2dee54d6b942f58650196582c` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:33.603939 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225ea91d-eecf-46f5-81d2-28b26404389c None admin] RBAC: Authorizing `identity:create_grant(project_id=346756b911774d55af2a9ad3de2143b4, user_id=4b5336d82912467b9cf6aa369a36d7e0, role_id=af257be2dee54d6b942f58650196582c)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:33.606681 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225ea91d-eecf-46f5-81d2-28b26404389c None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:33.646124 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97d6189-eda4-4b78-bac4-24cf7876f85d None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:33.694385 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 125/258] 10.222.0.55 () {58 vars in 1439 bytes} [Fri Sep 23 14:58:33 2022] PUT /identity/v3/projects/346756b911774d55af2a9ad3de2143b4/users/4b5336d82912467b9cf6aa369a36d7e0/roles/af257be2dee54d6b942f58650196582c => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 14:58:33.695608 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81ffb65-dfea-46c8-8b26-545217e3d62f None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:33.695900 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81ffb65-dfea-46c8-8b26-545217e3d62f None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:33.696227 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81ffb65-dfea-46c8-8b26-545217e3d62f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:33.701886 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852e2755-82c5-4c4a-bb37-0137d81bb312 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:33.702555 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852e2755-82c5-4c4a-bb37-0137d81bb312 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:33.702883 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852e2755-82c5-4c4a-bb37-0137d81bb312 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:33.703117 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852e2755-82c5-4c4a-bb37-0137d81bb312 None admin] PATH_INFO: `/v3/users/755ef177078341b6a1f30e126cb4a9fa` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:33.705746 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852e2755-82c5-4c4a-bb37-0137d81bb312 None admin] RBAC: Authorizing `identity:get_user(user_id=755ef177078341b6a1f30e126cb4a9fa)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:33.707212 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852e2755-82c5-4c4a-bb37-0137d81bb312 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:33.713121 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 134/259] 10.222.0.55 () {56 vars in 1177 bytes} [Fri Sep 23 14:58:33 2022] GET /identity/v3/users/755ef177078341b6a1f30e126cb4a9fa => generated 283 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:58:33.721275 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852e2755-82c5-4c4a-bb37-0137d81bb312 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:33.729867 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-b81ffb65-dfea-46c8-8b26-545217e3d62f None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 14:58:33.731639 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-b81ffb65-dfea-46c8-8b26-545217e3d62f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 14:58:33.779935 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:33.803433 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:33.856483 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 126/260] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Sep 23 14:58:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 14:58:33.920256 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852e2755-82c5-4c4a-bb37-0137d81bb312 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:33.980416 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9fed92-2527-4330-af86-77ec80529af5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:33.981058 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9fed92-2527-4330-af86-77ec80529af5 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:33.981339 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9fed92-2527-4330-af86-77ec80529af5 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:33.981633 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9fed92-2527-4330-af86-77ec80529af5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:34.072115 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9fed92-2527-4330-af86-77ec80529af5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:34.074430 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9fed92-2527-4330-af86-77ec80529af5 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:34.132501 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:34.186748 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 135/261] 10.222.0.55 () {58 vars in 1282 bytes} [Fri Sep 23 14:58:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 466 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 23 14:58:34.192389 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81ffb65-dfea-46c8-8b26-545217e3d62f None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:34.299690 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9fed92-2527-4330-af86-77ec80529af5 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:34.328181 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81ffb65-dfea-46c8-8b26-545217e3d62f None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:34.391367 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eda7463-cac4-49bb-99f5-2a49649ba6ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:34.392407 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eda7463-cac4-49bb-99f5-2a49649ba6ea None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:34.392889 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eda7463-cac4-49bb-99f5-2a49649ba6ea None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:34.393357 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eda7463-cac4-49bb-99f5-2a49649ba6ea None admin] PATH_INFO: `/v3/projects/346756b911774d55af2a9ad3de2143b4` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:34.395431 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eda7463-cac4-49bb-99f5-2a49649ba6ea None admin] RBAC: Authorizing `identity:get_project(project_id=346756b911774d55af2a9ad3de2143b4)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:34.397826 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eda7463-cac4-49bb-99f5-2a49649ba6ea None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:34.400633 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 127/262] 10.222.0.55 () {56 vars in 1186 bytes} [Fri Sep 23 14:58:34 2022] GET /identity/v3/projects/346756b911774d55af2a9ad3de2143b4 => generated 302 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:58:34.408759 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eda7463-cac4-49bb-99f5-2a49649ba6ea None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:34.468432 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:34.488035 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9fed92-2527-4330-af86-77ec80529af5 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:34.535191 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f46fbd1-afe2-4fb5-9243-54c7e685a3a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:34.535882 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f46fbd1-afe2-4fb5-9243-54c7e685a3a4 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:34.536203 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f46fbd1-afe2-4fb5-9243-54c7e685a3a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:34.536498 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f46fbd1-afe2-4fb5-9243-54c7e685a3a4 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:34.549287 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f46fbd1-afe2-4fb5-9243-54c7e685a3a4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 14:58:34.554356 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-7f46fbd1-afe2-4fb5-9243-54c7e685a3a4 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 23 14:58:34.554356 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 14:58:34.554356 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 14:58:34.554356 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 14:58:34.554356 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 14:58:34.554356 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 14:58:34.554356 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 14:58:34.554356 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 14:58:34.554356 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 14:58:34.554356 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 14:58:34.554356 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 14:58:34.554356 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 14:58:34.554356 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 14:58:34.554356 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 14:58:34.554356 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 14:58:34.554356 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 14:58:34.555214 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 14:58:34.555214 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 14:58:34.555214 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 14:58:34.555214 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 14:58:34.555214 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 14:58:34.555214 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 14:58:34.555214 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 14:58:34.555214 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 14:58:34.555214 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 14:58:34.555214 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 14:58:34.555214 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 14:58:34.555214 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Sep 23 14:58:34.555214 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 14:58:34.555214 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Sep 23 14:58:34.555214 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 14:58:34.555214 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 14:58:34.556602 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 14:58:34.556602 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Sep 23 14:58:34.556602 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 14:58:34.556602 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Sep 23 14:58:34.556602 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 14:58:34.556602 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 14:58:34.556602 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 14:58:34.556602 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 14:58:34.556602 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 14:58:34.556602 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 14:58:34.556602 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 23 14:58:34.556602 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Sep 23 14:58:34.556602 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 136/263] 10.222.0.55 () {56 vars in 1114 bytes} [Fri Sep 23 14:58:34 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 23 14:58:34.562891 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f46fbd1-afe2-4fb5-9243-54c7e685a3a4 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:34.596390 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:34.650153 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eda7463-cac4-49bb-99f5-2a49649ba6ea None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:34.677295 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f46fbd1-afe2-4fb5-9243-54c7e685a3a4 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:34.718016 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5599a4b3-6fc0-4d0d-829f-a816310a71db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:34.718230 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257747af-b5ac-4102-8f27-567d770163e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:34.718944 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257747af-b5ac-4102-8f27-567d770163e6 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:34.719232 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5599a4b3-6fc0-4d0d-829f-a816310a71db None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:34.719355 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257747af-b5ac-4102-8f27-567d770163e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:34.719719 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257747af-b5ac-4102-8f27-567d770163e6 None admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:34.719917 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5599a4b3-6fc0-4d0d-829f-a816310a71db None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:34.720416 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5599a4b3-6fc0-4d0d-829f-a816310a71db None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:34.720719 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257747af-b5ac-4102-8f27-567d770163e6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:58:34.721086 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257747af-b5ac-4102-8f27-567d770163e6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:34.721640 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5599a4b3-6fc0-4d0d-829f-a816310a71db None admin] RBAC: Adding query filter params (role.id=8e2919e79df040e8926f21607818b58c, scope.project.id=346756b911774d55af2a9ad3de2143b4, user.id=755ef177078341b6a1f30e126cb4a9fa) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:58:34.722093 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5599a4b3-6fc0-4d0d-829f-a816310a71db None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:34.722709 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257747af-b5ac-4102-8f27-567d770163e6 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:34.723831 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5599a4b3-6fc0-4d0d-829f-a816310a71db None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:34.732773 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 137/264] 10.222.0.55 () {56 vars in 1111 bytes} [Fri Sep 23 14:58:34 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 14:58:34.735146 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 128/265] 10.222.0.55 () {56 vars in 1374 bytes} [Fri Sep 23 14:58:34 2022] GET /identity/v3/role_assignments?user.id=755ef177078341b6a1f30e126cb4a9fa&scope.project.id=346756b911774d55af2a9ad3de2143b4&role.id=8e2919e79df040e8926f21607818b58c => generated 608 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:58:34.755476 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257747af-b5ac-4102-8f27-567d770163e6 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:34.860399 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:34.896269 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257747af-b5ac-4102-8f27-567d770163e6 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:34.937636 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e74c0ae-d3c7-472e-80dd-2f3ce8e6ebe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:34.938300 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e74c0ae-d3c7-472e-80dd-2f3ce8e6ebe2 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:34.938582 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e74c0ae-d3c7-472e-80dd-2f3ce8e6ebe2 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:34.938857 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e74c0ae-d3c7-472e-80dd-2f3ce8e6ebe2 None admin] PATH_INFO: `/v3/users/755ef177078341b6a1f30e126cb4a9fa` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:34.940056 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e74c0ae-d3c7-472e-80dd-2f3ce8e6ebe2 None admin] RBAC: Authorizing `identity:get_user(user_id=755ef177078341b6a1f30e126cb4a9fa)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:34.941530 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e74c0ae-d3c7-472e-80dd-2f3ce8e6ebe2 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:34.943412 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 138/266] 10.222.0.55 () {56 vars in 1177 bytes} [Fri Sep 23 14:58:34 2022] GET /identity/v3/users/755ef177078341b6a1f30e126cb4a9fa => generated 283 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:58:34.949931 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5599a4b3-6fc0-4d0d-829f-a816310a71db None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:35.094872 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:35.105834 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5599a4b3-6fc0-4d0d-829f-a816310a71db None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:35.143311 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274df1dc-c860-49a1-9b5e-93ec823487ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:35.143939 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274df1dc-c860-49a1-9b5e-93ec823487ab None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:35.144253 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274df1dc-c860-49a1-9b5e-93ec823487ab None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:35.144532 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274df1dc-c860-49a1-9b5e-93ec823487ab None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:35.207292 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d9ff20-3e4c-4421-a225-4850405a2f8d None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:35.207786 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d9ff20-3e4c-4421-a225-4850405a2f8d None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:35.208017 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d9ff20-3e4c-4421-a225-4850405a2f8d None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:35.209411 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 139/267] 10.222.0.55 () {54 vars in 927 bytes} [Fri Sep 23 14:58:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 23 14:58:35.215952 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf41621c-fe41-4ab2-85ee-41774003cec7 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:35.216555 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf41621c-fe41-4ab2-85ee-41774003cec7 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:35.216555 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf41621c-fe41-4ab2-85ee-41774003cec7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:35.225893 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274df1dc-c860-49a1-9b5e-93ec823487ab None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:35.227384 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274df1dc-c860-49a1-9b5e-93ec823487ab None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:35.260978 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-cf41621c-fe41-4ab2-85ee-41774003cec7 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 14:58:35.262257 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-cf41621c-fe41-4ab2-85ee-41774003cec7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 14:58:35.344457 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:35.351253 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 129/268] 10.222.0.55 () {58 vars in 1282 bytes} [Fri Sep 23 14:58:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 23 14:58:35.357768 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274df1dc-c860-49a1-9b5e-93ec823487ab None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:35.452674 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 140/269] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Sep 23 14:58:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 14:58:35.487459 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274df1dc-c860-49a1-9b5e-93ec823487ab None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:35.528744 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c56aa7-b805-410d-877c-22c30a3f3f1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:35.529394 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c56aa7-b805-410d-877c-22c30a3f3f1d None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:35.529678 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c56aa7-b805-410d-877c-22c30a3f3f1d None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:35.529956 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c56aa7-b805-410d-877c-22c30a3f3f1d None admin] PATH_INFO: `/v3/projects/346756b911774d55af2a9ad3de2143b4` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:35.531178 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c56aa7-b805-410d-877c-22c30a3f3f1d None admin] RBAC: Authorizing `identity:get_project(project_id=346756b911774d55af2a9ad3de2143b4)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:35.532930 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c56aa7-b805-410d-877c-22c30a3f3f1d None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:35.534859 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 130/270] 10.222.0.55 () {56 vars in 1186 bytes} [Fri Sep 23 14:58:35 2022] GET /identity/v3/projects/346756b911774d55af2a9ad3de2143b4 => generated 302 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 14:58:35.541422 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf41621c-fe41-4ab2-85ee-41774003cec7 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:35.644462 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:35.683866 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf41621c-fe41-4ab2-85ee-41774003cec7 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:35.718547 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82e668e-695f-48fb-8fd2-571b969484af None None] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:35.718922 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82e668e-695f-48fb-8fd2-571b969484af None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:35.719276 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82e668e-695f-48fb-8fd2-571b969484af None None] PATH_INFO: `/` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:35.720878 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 131/271] 10.222.0.55 () {54 vars in 927 bytes} [Fri Sep 23 14:58:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 23 14:58:35.725759 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd097890-094b-49a8-bfa4-c1b7916e7780 None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:35.726047 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd097890-094b-49a8-bfa4-c1b7916e7780 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:35.726324 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd097890-094b-49a8-bfa4-c1b7916e7780 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:35.735339 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982c03a5-e623-4f19-9748-2c8f418fc832 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:35.735957 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982c03a5-e623-4f19-9748-2c8f418fc832 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:35.737065 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982c03a5-e623-4f19-9748-2c8f418fc832 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:35.737345 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982c03a5-e623-4f19-9748-2c8f418fc832 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:35.738082 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982c03a5-e623-4f19-9748-2c8f418fc832 None admin] RBAC: Adding query filter params (role.id=5c74fdc0e9cc44538cd8282006ed4443, scope.project.id=346756b911774d55af2a9ad3de2143b4, user.id=755ef177078341b6a1f30e126cb4a9fa) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:58:35.738394 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982c03a5-e623-4f19-9748-2c8f418fc832 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:35.739827 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982c03a5-e623-4f19-9748-2c8f418fc832 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:35.754830 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 141/272] 10.222.0.55 () {56 vars in 1374 bytes} [Fri Sep 23 14:58:35 2022] GET /identity/v3/role_assignments?user.id=755ef177078341b6a1f30e126cb4a9fa&scope.project.id=346756b911774d55af2a9ad3de2143b4&role.id=5c74fdc0e9cc44538cd8282006ed4443 => generated 608 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:58:35.756420 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-dd097890-094b-49a8-bfa4-c1b7916e7780 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 14:58:35.757643 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-dd097890-094b-49a8-bfa4-c1b7916e7780 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 14:58:35.819674 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:35.831541 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 132/273] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Sep 23 14:58:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 23 14:58:35.894798 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982c03a5-e623-4f19-9748-2c8f418fc832 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:36.010616 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982c03a5-e623-4f19-9748-2c8f418fc832 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:36.049052 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03da2b28-6d39-459b-ac47-a5befa10d75e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:36.050151 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03da2b28-6d39-459b-ac47-a5befa10d75e None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:36.050640 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03da2b28-6d39-459b-ac47-a5befa10d75e None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:36.051152 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03da2b28-6d39-459b-ac47-a5befa10d75e None admin] PATH_INFO: `/v3/domains/default` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:36.052552 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03da2b28-6d39-459b-ac47-a5befa10d75e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:36.054236 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03da2b28-6d39-459b-ac47-a5befa10d75e None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:36.057265 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 142/274] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Sep 23 14:58:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:58:36.066152 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd097890-094b-49a8-bfa4-c1b7916e7780 None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:36.123591 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03da2b28-6d39-459b-ac47-a5befa10d75e None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:36.144352 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:36.205773 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd097890-094b-49a8-bfa4-c1b7916e7780 None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:36.248134 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03da2b28-6d39-459b-ac47-a5befa10d75e None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:36.254202 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8cb1a7-b9b7-4601-928c-d4c8757dc934 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:36.254833 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8cb1a7-b9b7-4601-928c-d4c8757dc934 None admin] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:36.255112 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8cb1a7-b9b7-4601-928c-d4c8757dc934 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:36.255385 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8cb1a7-b9b7-4601-928c-d4c8757dc934 None admin] PATH_INFO: `/v3/users` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:36.256510 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8cb1a7-b9b7-4601-928c-d4c8757dc934 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:36.257990 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8cb1a7-b9b7-4601-928c-d4c8757dc934 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:36.292900 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ed5c65-5874-433a-83e9-f0780b9eefed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:36.298989 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ed5c65-5874-433a-83e9-f0780b9eefed None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:36.298989 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ed5c65-5874-433a-83e9-f0780b9eefed None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:36.298989 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ed5c65-5874-433a-83e9-f0780b9eefed None admin] PATH_INFO: `/v3/roles/member` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:36.305680 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ed5c65-5874-433a-83e9-f0780b9eefed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 14:58:36.312790 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-34ed5c65-5874-433a-83e9-f0780b9eefed None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 23 14:58:36.312790 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 14:58:36.312790 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 14:58:36.312790 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 14:58:36.312790 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 14:58:36.312790 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 14:58:36.312790 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 14:58:36.312790 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 14:58:36.312790 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 14:58:36.312790 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 14:58:36.312790 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 14:58:36.312790 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 14:58:36.312790 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 14:58:36.312790 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 14:58:36.312790 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 14:58:36.312790 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 14:58:36.313738 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 14:58:36.313738 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 14:58:36.313738 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 14:58:36.313738 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 14:58:36.313738 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 14:58:36.313738 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 14:58:36.313738 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 14:58:36.313738 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 14:58:36.313738 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 14:58:36.313738 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 14:58:36.313738 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 14:58:36.313738 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Sep 23 14:58:36.313738 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 14:58:36.313738 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Sep 23 14:58:36.313738 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 14:58:36.313738 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 14:58:36.314575 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 14:58:36.314575 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Sep 23 14:58:36.314575 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 14:58:36.314575 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Sep 23 14:58:36.314575 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 14:58:36.314575 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 14:58:36.314575 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 14:58:36.314575 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 14:58:36.314575 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 14:58:36.314575 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 14:58:36.314575 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 23 14:58:36.314575 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Sep 23 14:58:36.314575 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 143/275] 10.222.0.55 () {56 vars in 1099 bytes} [Fri Sep 23 14:58:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 23 14:58:36.321988 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ed5c65-5874-433a-83e9-f0780b9eefed None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:36.393344 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:36.414105 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 133/276] 10.222.0.55 () {60 vars in 1132 bytes} [Fri Sep 23 14:58:36 2022] POST /identity/v3/users => generated 293 bytes in 349 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 14:58:36.453917 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ed5c65-5874-433a-83e9-f0780b9eefed None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:36.494876 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd45c8f6-3074-4ed4-b882-df5620fc3592 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:36.495502 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd45c8f6-3074-4ed4-b882-df5620fc3592 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:36.495835 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd45c8f6-3074-4ed4-b882-df5620fc3592 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:36.496185 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd45c8f6-3074-4ed4-b882-df5620fc3592 None admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:36.496867 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd45c8f6-3074-4ed4-b882-df5620fc3592 None admin] RBAC: Adding query filter params (name=member) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:58:36.497224 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd45c8f6-3074-4ed4-b882-df5620fc3592 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:36.498752 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd45c8f6-3074-4ed4-b882-df5620fc3592 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:36.509112 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 144/277] 10.222.0.55 () {56 vars in 1101 bytes} [Fri Sep 23 14:58:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:58:36.515289 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8cb1a7-b9b7-4601-928c-d4c8757dc934 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:36.574031 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:36.640367 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8cb1a7-b9b7-4601-928c-d4c8757dc934 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:36.686076 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5dd176-28ec-49e9-929e-ee2013253e01 None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:36.686357 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5dd176-28ec-49e9-929e-ee2013253e01 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:36.686553 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09e6e9d-047c-4c3a-94f2-ffc152d2ff04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:36.686679 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5dd176-28ec-49e9-929e-ee2013253e01 None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:36.687164 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09e6e9d-047c-4c3a-94f2-ffc152d2ff04 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:36.687448 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09e6e9d-047c-4c3a-94f2-ffc152d2ff04 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:36.687728 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09e6e9d-047c-4c3a-94f2-ffc152d2ff04 None admin] PATH_INFO: `/v3/users/e501b2f5ce44459c900cbc120eb93931` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:36.688531 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 145/278] 10.222.0.55 () {54 vars in 927 bytes} [Fri Sep 23 14:58:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 23 14:58:36.695033 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1725e9-0701-4c96-b4f1-d596b466ad32 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:36.695372 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1725e9-0701-4c96-b4f1-d596b466ad32 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:36.695586 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1725e9-0701-4c96-b4f1-d596b466ad32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:36.706606 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09e6e9d-047c-4c3a-94f2-ffc152d2ff04 None admin] RBAC: Authorizing `identity:get_user(user_id=e501b2f5ce44459c900cbc120eb93931)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:36.707703 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09e6e9d-047c-4c3a-94f2-ffc152d2ff04 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:36.709749 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 134/279] 10.222.0.55 () {56 vars in 1177 bytes} [Fri Sep 23 14:58:36 2022] GET /identity/v3/users/e501b2f5ce44459c900cbc120eb93931 => generated 293 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:58:36.717118 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09e6e9d-047c-4c3a-94f2-ffc152d2ff04 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:36.719512 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-bc1725e9-0701-4c96-b4f1-d596b466ad32 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 14:58:36.720998 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-bc1725e9-0701-4c96-b4f1-d596b466ad32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 14:58:36.817169 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 146/280] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Sep 23 14:58:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 14:58:36.820507 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14686143-10d4-4ba0-b073-450102065ecd None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:36.820507 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14686143-10d4-4ba0-b073-450102065ecd None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:36.820507 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14686143-10d4-4ba0-b073-450102065ecd None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:36.820507 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 147/281] 10.222.0.55 () {54 vars in 927 bytes} [Fri Sep 23 14:58:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 14:58:36.827297 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43196788-1806-43c3-8163-b6d2025e2bcb None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:36.827583 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43196788-1806-43c3-8163-b6d2025e2bcb None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:36.827856 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43196788-1806-43c3-8163-b6d2025e2bcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:36.850110 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-43196788-1806-43c3-8163-b6d2025e2bcb None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 14:58:36.851495 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-43196788-1806-43c3-8163-b6d2025e2bcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 14:58:36.858049 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09e6e9d-047c-4c3a-94f2-ffc152d2ff04 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:36.861046 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:36.898504 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3824ce08-2646-480f-bd89-b8e117758036 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:36.899934 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3824ce08-2646-480f-bd89-b8e117758036 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:36.900374 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3824ce08-2646-480f-bd89-b8e117758036 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:36.900659 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3824ce08-2646-480f-bd89-b8e117758036 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:36.901401 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3824ce08-2646-480f-bd89-b8e117758036 None admin] RBAC: Adding query filter params (role.id=af257be2dee54d6b942f58650196582c, user.id=e501b2f5ce44459c900cbc120eb93931, scope.system=all) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:58:36.902095 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3824ce08-2646-480f-bd89-b8e117758036 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:36.904024 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3824ce08-2646-480f-bd89-b8e117758036 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:36.911496 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 135/282] 10.222.0.55 () {56 vars in 1308 bytes} [Fri Sep 23 14:58:36 2022] GET /identity/v3/role_assignments?user.id=e501b2f5ce44459c900cbc120eb93931&scope.system=all&role.id=af257be2dee54d6b942f58650196582c => generated 226 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 14:58:36.913194 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:36.937877 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 148/283] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Sep 23 14:58:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 14:58:37.144905 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3824ce08-2646-480f-bd89-b8e117758036 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:37.244938 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43196788-1806-43c3-8163-b6d2025e2bcb None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:37.259379 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3824ce08-2646-480f-bd89-b8e117758036 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:37.322789 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2632b28d-beff-4a76-a1b1-4fa1d86066a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:37.323433 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2632b28d-beff-4a76-a1b1-4fa1d86066a8 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:37.323712 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2632b28d-beff-4a76-a1b1-4fa1d86066a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:37.323998 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2632b28d-beff-4a76-a1b1-4fa1d86066a8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:37.341611 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2632b28d-beff-4a76-a1b1-4fa1d86066a8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 14:58:37.346505 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application [None req-2632b28d-beff-4a76-a1b1-4fa1d86066a8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 23 14:58:37.346505 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 14:58:37.346505 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 14:58:37.346505 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 14:58:37.346505 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 14:58:37.346505 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 14:58:37.346505 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 14:58:37.346505 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 14:58:37.346505 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 14:58:37.346505 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 14:58:37.346505 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 14:58:37.346505 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 14:58:37.346505 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 14:58:37.346505 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 14:58:37.346505 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 14:58:37.346505 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 14:58:37.347374 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 14:58:37.347374 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 14:58:37.347374 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 14:58:37.347374 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 14:58:37.347374 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 14:58:37.347374 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 14:58:37.347374 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 14:58:37.347374 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 14:58:37.347374 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 14:58:37.347374 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 14:58:37.347374 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 14:58:37.347374 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application with Lock( Sep 23 14:58:37.347374 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 14:58:37.347374 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._enter() Sep 23 14:58:37.347374 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 14:58:37.347374 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 14:58:37.348186 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 14:58:37.348186 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.creator() Sep 23 14:58:37.348186 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 14:58:37.348186 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application created_value = creator( Sep 23 14:58:37.348186 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 14:58:37.348186 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 14:58:37.348186 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 14:58:37.348186 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 14:58:37.348186 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 14:58:37.348186 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 14:58:37.348186 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 23 14:58:37.348186 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Sep 23 14:58:37.348186 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 136/284] 10.222.0.55 () {56 vars in 1099 bytes} [Fri Sep 23 14:58:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 14:58:37.353733 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2632b28d-beff-4a76-a1b1-4fa1d86066a8 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:37.398123 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43196788-1806-43c3-8163-b6d2025e2bcb None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:37.399190 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:37.440118 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc942196-f864-4a63-8c60-4adef661b2cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:37.440767 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc942196-f864-4a63-8c60-4adef661b2cd None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:37.441049 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc942196-f864-4a63-8c60-4adef661b2cd None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:37.441322 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc942196-f864-4a63-8c60-4adef661b2cd None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:37.453108 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc942196-f864-4a63-8c60-4adef661b2cd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 14:58:37.459804 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-cc942196-f864-4a63-8c60-4adef661b2cd None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 23 14:58:37.459804 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 14:58:37.459804 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 14:58:37.459804 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 14:58:37.459804 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 14:58:37.459804 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 14:58:37.459804 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 14:58:37.459804 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 14:58:37.459804 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 14:58:37.459804 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 14:58:37.459804 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 14:58:37.459804 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 14:58:37.459804 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 14:58:37.459804 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 14:58:37.459804 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 14:58:37.459804 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 14:58:37.460704 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 14:58:37.460704 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 14:58:37.460704 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 14:58:37.460704 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 14:58:37.460704 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 14:58:37.460704 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 14:58:37.460704 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 14:58:37.460704 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 14:58:37.460704 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 14:58:37.460704 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 14:58:37.460704 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 14:58:37.460704 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Sep 23 14:58:37.460704 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 14:58:37.460704 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Sep 23 14:58:37.460704 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 14:58:37.460704 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 14:58:37.461489 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 14:58:37.461489 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Sep 23 14:58:37.461489 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 14:58:37.461489 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Sep 23 14:58:37.461489 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 14:58:37.461489 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 14:58:37.461489 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 14:58:37.461489 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 14:58:37.461489 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 14:58:37.461489 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 14:58:37.461489 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 23 14:58:37.461489 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Sep 23 14:58:37.461489 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 149/285] 10.222.0.55 () {56 vars in 1099 bytes} [Fri Sep 23 14:58:37 2022] GET /identity/v3/roles/reader => generated 84 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 14:58:37.467685 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc942196-f864-4a63-8c60-4adef661b2cd None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:37.522638 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:37.578101 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2632b28d-beff-4a76-a1b1-4fa1d86066a8 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:37.580071 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc942196-f864-4a63-8c60-4adef661b2cd None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:37.617167 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa8456c-e4ab-419d-8e13-90f1b50fa063 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:37.617810 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa8456c-e4ab-419d-8e13-90f1b50fa063 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:37.618199 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa8456c-e4ab-419d-8e13-90f1b50fa063 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:37.624002 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa8456c-e4ab-419d-8e13-90f1b50fa063 None admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:37.624002 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa8456c-e4ab-419d-8e13-90f1b50fa063 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:58:37.624002 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa8456c-e4ab-419d-8e13-90f1b50fa063 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:37.624002 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa8456c-e4ab-419d-8e13-90f1b50fa063 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:37.631517 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 150/286] 10.222.0.55 () {56 vars in 1101 bytes} [Fri Sep 23 14:58:37 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:58:37.641352 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa8456c-e4ab-419d-8e13-90f1b50fa063 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:37.675862 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc5960a-4f15-4bb6-a9c9-ef78f679edb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:37.676631 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc5960a-4f15-4bb6-a9c9-ef78f679edb3 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:37.676912 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc5960a-4f15-4bb6-a9c9-ef78f679edb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:37.677183 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc5960a-4f15-4bb6-a9c9-ef78f679edb3 None admin] PATH_INFO: `/v3/roles` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:37.677907 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc5960a-4f15-4bb6-a9c9-ef78f679edb3 None admin] RBAC: Adding query filter params (name=member) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:58:37.678311 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc5960a-4f15-4bb6-a9c9-ef78f679edb3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:37.679872 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc5960a-4f15-4bb6-a9c9-ef78f679edb3 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:37.695387 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 137/287] 10.222.0.55 () {56 vars in 1101 bytes} [Fri Sep 23 14:58:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 342 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:58:37.709008 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc5960a-4f15-4bb6-a9c9-ef78f679edb3 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:37.760043 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa8456c-e4ab-419d-8e13-90f1b50fa063 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:37.772405 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:37.821792 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1312be96-a51c-4b7c-8d6c-c9fed430145d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:37.822441 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1312be96-a51c-4b7c-8d6c-c9fed430145d None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:37.822721 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1312be96-a51c-4b7c-8d6c-c9fed430145d None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:37.822991 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1312be96-a51c-4b7c-8d6c-c9fed430145d None admin] PATH_INFO: `/v3/users/3a099085ae5b4b5780b0f3c4d86b19f0` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:37.824236 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1312be96-a51c-4b7c-8d6c-c9fed430145d None admin] RBAC: Authorizing `identity:get_user(user_id=3a099085ae5b4b5780b0f3c4d86b19f0)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:37.825927 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1312be96-a51c-4b7c-8d6c-c9fed430145d None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:37.827900 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 151/288] 10.222.0.55 () {56 vars in 1177 bytes} [Fri Sep 23 14:58:37 2022] GET /identity/v3/users/3a099085ae5b4b5780b0f3c4d86b19f0 => generated 297 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:58:37.833325 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1312be96-a51c-4b7c-8d6c-c9fed430145d None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:37.912506 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:37.924310 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc5960a-4f15-4bb6-a9c9-ef78f679edb3 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:37.983330 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635c39cc-36ba-4f70-a416-1682e1917a37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:37.983993 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635c39cc-36ba-4f70-a416-1682e1917a37 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:37.984283 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635c39cc-36ba-4f70-a416-1682e1917a37 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:37.984556 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635c39cc-36ba-4f70-a416-1682e1917a37 None admin] PATH_INFO: `/v3/users/4b5336d82912467b9cf6aa369a36d7e0` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:37.985828 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635c39cc-36ba-4f70-a416-1682e1917a37 None admin] RBAC: Authorizing `identity:get_user(user_id=4b5336d82912467b9cf6aa369a36d7e0)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:37.987266 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635c39cc-36ba-4f70-a416-1682e1917a37 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:37.989018 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 138/289] 10.222.0.55 () {56 vars in 1177 bytes} [Fri Sep 23 14:58:37 2022] GET /identity/v3/users/4b5336d82912467b9cf6aa369a36d7e0 => generated 297 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:58:37.996845 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635c39cc-36ba-4f70-a416-1682e1917a37 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:38.045224 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1312be96-a51c-4b7c-8d6c-c9fed430145d None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:38.107990 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5db731b-5a98-476d-9778-27a7faf0a640 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:38.108650 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5db731b-5a98-476d-9778-27a7faf0a640 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:38.108933 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5db731b-5a98-476d-9778-27a7faf0a640 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:38.109208 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5db731b-5a98-476d-9778-27a7faf0a640 None admin] PATH_INFO: `/v3/projects/346756b911774d55af2a9ad3de2143b4` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:38.112708 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5db731b-5a98-476d-9778-27a7faf0a640 None admin] RBAC: Authorizing `identity:get_project(project_id=346756b911774d55af2a9ad3de2143b4)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:38.114265 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5db731b-5a98-476d-9778-27a7faf0a640 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:38.116233 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 152/290] 10.222.0.55 () {56 vars in 1186 bytes} [Fri Sep 23 14:58:37 2022] GET /identity/v3/projects/346756b911774d55af2a9ad3de2143b4 => generated 302 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:58:38.122175 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5db731b-5a98-476d-9778-27a7faf0a640 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:38.156196 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635c39cc-36ba-4f70-a416-1682e1917a37 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:38.200130 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b3f855-7422-429f-9957-9b72322a45fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:38.201200 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b3f855-7422-429f-9957-9b72322a45fc None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:38.201694 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b3f855-7422-429f-9957-9b72322a45fc None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:38.202161 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b3f855-7422-429f-9957-9b72322a45fc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:38.207066 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:38.231368 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:38.273638 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b3f855-7422-429f-9957-9b72322a45fc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:38.275450 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b3f855-7422-429f-9957-9b72322a45fc None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:38.300484 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5db731b-5a98-476d-9778-27a7faf0a640 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:38.364082 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73f24ca-b20a-4dda-a009-b4b2c39f05a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:38.366553 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73f24ca-b20a-4dda-a009-b4b2c39f05a0 None admin] REQUEST_METHOD: `PUT` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:38.367752 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73f24ca-b20a-4dda-a009-b4b2c39f05a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:38.368354 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73f24ca-b20a-4dda-a009-b4b2c39f05a0 None admin] PATH_INFO: `/v3/projects/346756b911774d55af2a9ad3de2143b4/users/3a099085ae5b4b5780b0f3c4d86b19f0/roles/ea002840090d4194bb75fa083a55dc5f` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:38.372772 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73f24ca-b20a-4dda-a009-b4b2c39f05a0 None admin] RBAC: Authorizing `identity:create_grant(project_id=346756b911774d55af2a9ad3de2143b4, user_id=3a099085ae5b4b5780b0f3c4d86b19f0, role_id=ea002840090d4194bb75fa083a55dc5f)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:38.375202 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73f24ca-b20a-4dda-a009-b4b2c39f05a0 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:38.421271 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 139/291] 10.222.0.55 () {58 vars in 1282 bytes} [Fri Sep 23 14:58:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 23 14:58:38.427086 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b3f855-7422-429f-9957-9b72322a45fc None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:38.460073 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 153/292] 10.222.0.55 () {58 vars in 1439 bytes} [Fri Sep 23 14:58:38 2022] PUT /identity/v3/projects/346756b911774d55af2a9ad3de2143b4/users/3a099085ae5b4b5780b0f3c4d86b19f0/roles/ea002840090d4194bb75fa083a55dc5f => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 14:58:38.495970 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:38.543251 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b3f855-7422-429f-9957-9b72322a45fc None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:38.591605 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead9c2a3-c801-4c20-a60d-c8d7ac50d778 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:38.592286 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead9c2a3-c801-4c20-a60d-c8d7ac50d778 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:38.592568 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead9c2a3-c801-4c20-a60d-c8d7ac50d778 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:38.592846 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead9c2a3-c801-4c20-a60d-c8d7ac50d778 None admin] PATH_INFO: `/v3/projects/346756b911774d55af2a9ad3de2143b4` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:38.593986 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead9c2a3-c801-4c20-a60d-c8d7ac50d778 None admin] RBAC: Authorizing `identity:get_project(project_id=346756b911774d55af2a9ad3de2143b4)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:38.595455 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead9c2a3-c801-4c20-a60d-c8d7ac50d778 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:38.597123 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 140/293] 10.222.0.55 () {56 vars in 1186 bytes} [Fri Sep 23 14:58:38 2022] GET /identity/v3/projects/346756b911774d55af2a9ad3de2143b4 => generated 302 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 14:58:38.603423 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73f24ca-b20a-4dda-a009-b4b2c39f05a0 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:38.758137 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73f24ca-b20a-4dda-a009-b4b2c39f05a0 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:38.760368 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:38.796467 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50260ec4-fe12-4f46-b8bb-b03c365baa3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:38.797561 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50260ec4-fe12-4f46-b8bb-b03c365baa3a None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:38.797881 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50260ec4-fe12-4f46-b8bb-b03c365baa3a None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:38.798179 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50260ec4-fe12-4f46-b8bb-b03c365baa3a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:38.798916 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50260ec4-fe12-4f46-b8bb-b03c365baa3a None admin] RBAC: Adding query filter params (role.id=af257be2dee54d6b942f58650196582c, scope.project.id=346756b911774d55af2a9ad3de2143b4, user.id=4b5336d82912467b9cf6aa369a36d7e0) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:58:38.799229 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50260ec4-fe12-4f46-b8bb-b03c365baa3a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:38.800712 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50260ec4-fe12-4f46-b8bb-b03c365baa3a None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:38.808249 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 154/294] 10.222.0.55 () {56 vars in 1374 bytes} [Fri Sep 23 14:58:38 2022] GET /identity/v3/role_assignments?user.id=4b5336d82912467b9cf6aa369a36d7e0&scope.project.id=346756b911774d55af2a9ad3de2143b4&role.id=af257be2dee54d6b942f58650196582c => generated 608 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:58:39.247224 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951ee351-d128-4d0a-8006-e385daf880b2 None None] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:39.247900 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951ee351-d128-4d0a-8006-e385daf880b2 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:39.248458 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951ee351-d128-4d0a-8006-e385daf880b2 None None] PATH_INFO: `/` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:39.250008 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 141/295] 10.222.0.55 () {54 vars in 927 bytes} [Fri Sep 23 14:58:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 14:58:39.256439 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52de96e-0ad1-48ea-a61a-e3c79b0c8aa5 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:39.257007 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52de96e-0ad1-48ea-a61a-e3c79b0c8aa5 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:39.257538 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52de96e-0ad1-48ea-a61a-e3c79b0c8aa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:39.278905 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-a52de96e-0ad1-48ea-a61a-e3c79b0c8aa5 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 14:58:39.280246 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-a52de96e-0ad1-48ea-a61a-e3c79b0c8aa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 14:58:39.378930 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 155/296] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Sep 23 14:58:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 14:58:39.460088 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f034121-480a-4ea6-bd53-5e8012540af7 None None] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:39.460705 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f034121-480a-4ea6-bd53-5e8012540af7 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:39.461174 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f034121-480a-4ea6-bd53-5e8012540af7 None None] PATH_INFO: `/` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:39.463687 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 142/297] 10.222.0.55 () {54 vars in 927 bytes} [Fri Sep 23 14:58:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 14:58:39.469168 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0dec28-64b5-4500-b5ce-4c38ce9fa1f0 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:39.469612 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0dec28-64b5-4500-b5ce-4c38ce9fa1f0 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:39.469783 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0dec28-64b5-4500-b5ce-4c38ce9fa1f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:39.492032 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-da0dec28-64b5-4500-b5ce-4c38ce9fa1f0 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 14:58:39.493831 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-da0dec28-64b5-4500-b5ce-4c38ce9fa1f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 14:58:39.569639 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 156/298] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Sep 23 14:58:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 14:58:39.623002 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08561311-409e-46ab-872b-e880591439db None None] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:39.624007 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08561311-409e-46ab-872b-e880591439db None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:39.624491 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08561311-409e-46ab-872b-e880591439db None None] PATH_INFO: `/` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:39.626499 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 143/299] 10.222.0.55 () {54 vars in 927 bytes} [Fri Sep 23 14:58:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 14:58:39.632609 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d40c7cf-1457-420c-a2b2-7cde141cf867 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:39.633188 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d40c7cf-1457-420c-a2b2-7cde141cf867 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:39.633667 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d40c7cf-1457-420c-a2b2-7cde141cf867 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:39.653584 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-5d40c7cf-1457-420c-a2b2-7cde141cf867 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 14:58:39.655127 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-5d40c7cf-1457-420c-a2b2-7cde141cf867 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 14:58:39.676863 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:39.688929 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08561311-409e-46ab-872b-e880591439db None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:39.735043 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 157/300] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Sep 23 14:58:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 14:58:39.802309 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08561311-409e-46ab-872b-e880591439db None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:39.828665 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:39.837358 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044c78cb-973c-4d69-8529-313768e371ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:39.837993 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044c78cb-973c-4d69-8529-313768e371ab None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:39.838267 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044c78cb-973c-4d69-8529-313768e371ab None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:39.838592 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044c78cb-973c-4d69-8529-313768e371ab None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:39.849303 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044c78cb-973c-4d69-8529-313768e371ab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 14:58:39.853725 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d40c7cf-1457-420c-a2b2-7cde141cf867 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:39.854948 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application [None req-044c78cb-973c-4d69-8529-313768e371ab None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 23 14:58:39.854948 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 14:58:39.854948 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 14:58:39.854948 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 14:58:39.854948 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 14:58:39.854948 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 14:58:39.854948 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 14:58:39.854948 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 14:58:39.854948 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 14:58:39.854948 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 14:58:39.854948 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 14:58:39.854948 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 14:58:39.854948 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 14:58:39.854948 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 14:58:39.854948 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 14:58:39.854948 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 14:58:39.855805 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 14:58:39.855805 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 14:58:39.855805 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 14:58:39.855805 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 14:58:39.855805 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 14:58:39.855805 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 14:58:39.855805 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 14:58:39.855805 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 14:58:39.855805 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 14:58:39.855805 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 14:58:39.855805 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 14:58:39.855805 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application with Lock( Sep 23 14:58:39.855805 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 14:58:39.855805 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._enter() Sep 23 14:58:39.855805 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 14:58:39.855805 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 14:58:39.856637 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 14:58:39.856637 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.creator() Sep 23 14:58:39.856637 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 14:58:39.856637 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application created_value = creator( Sep 23 14:58:39.856637 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 14:58:39.856637 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 14:58:39.856637 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 14:58:39.856637 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 14:58:39.856637 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 14:58:39.856637 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 14:58:39.856637 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 23 14:58:39.856637 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Sep 23 14:58:39.856637 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 144/301] 10.222.0.55 () {56 vars in 1099 bytes} [Fri Sep 23 14:58:39 2022] GET /identity/v3/roles/reader => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 23 14:58:39.869546 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044c78cb-973c-4d69-8529-313768e371ab None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:39.981998 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d40c7cf-1457-420c-a2b2-7cde141cf867 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:39.999511 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044c78cb-973c-4d69-8529-313768e371ab None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:40.023613 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb2a42e-8103-4cab-9997-8b257972912e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:40.025232 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb2a42e-8103-4cab-9997-8b257972912e None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:40.025727 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb2a42e-8103-4cab-9997-8b257972912e None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:40.026232 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb2a42e-8103-4cab-9997-8b257972912e None admin] PATH_INFO: `/v3/domains/default` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:40.027710 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb2a42e-8103-4cab-9997-8b257972912e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:40.030312 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb2a42e-8103-4cab-9997-8b257972912e None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:40.032669 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 158/302] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Sep 23 14:58:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 14:58:40.033677 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb2a42e-8103-4cab-9997-8b257972912e None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:40.041773 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fe8db8-fc88-4b6d-9ae1-04c624e07194 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:40.042398 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fe8db8-fc88-4b6d-9ae1-04c624e07194 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:40.042707 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fe8db8-fc88-4b6d-9ae1-04c624e07194 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:40.042984 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fe8db8-fc88-4b6d-9ae1-04c624e07194 None admin] PATH_INFO: `/v3/roles` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:40.043652 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fe8db8-fc88-4b6d-9ae1-04c624e07194 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:58:40.043959 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fe8db8-fc88-4b6d-9ae1-04c624e07194 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:40.045590 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fe8db8-fc88-4b6d-9ae1-04c624e07194 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:40.055083 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 145/303] 10.222.0.55 () {56 vars in 1101 bytes} [Fri Sep 23 14:58:39 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:58:40.056930 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fe8db8-fc88-4b6d-9ae1-04c624e07194 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:40.081693 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:40.109038 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:40.161819 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb2a42e-8103-4cab-9997-8b257972912e None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:40.168882 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fe8db8-fc88-4b6d-9ae1-04c624e07194 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:40.198637 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbb47b5-9a30-49e6-96a9-f10a42ea930f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:40.199277 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbb47b5-9a30-49e6-96a9-f10a42ea930f None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:40.199564 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbb47b5-9a30-49e6-96a9-f10a42ea930f None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:40.199841 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbb47b5-9a30-49e6-96a9-f10a42ea930f None admin] PATH_INFO: `/v3/roles/member` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:40.203792 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a6e7fa-a4fe-4cb8-b188-839463d87124 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:40.204434 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a6e7fa-a4fe-4cb8-b188-839463d87124 None admin] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:40.204710 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a6e7fa-a4fe-4cb8-b188-839463d87124 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:40.204980 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a6e7fa-a4fe-4cb8-b188-839463d87124 None admin] PATH_INFO: `/v3/groups` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:40.205606 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a6e7fa-a4fe-4cb8-b188-839463d87124 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:40.207049 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a6e7fa-a4fe-4cb8-b188-839463d87124 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:40.210838 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbb47b5-9a30-49e6-96a9-f10a42ea930f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 14:58:40.216397 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-6bbb47b5-9a30-49e6-96a9-f10a42ea930f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 23 14:58:40.216397 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 14:58:40.216397 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 14:58:40.216397 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 14:58:40.216397 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 14:58:40.216397 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 14:58:40.216397 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 14:58:40.216397 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 14:58:40.216397 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 14:58:40.216397 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 14:58:40.216397 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 14:58:40.216397 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 14:58:40.216397 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 14:58:40.216397 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 14:58:40.216397 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 14:58:40.216397 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 14:58:40.217509 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 14:58:40.217509 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 14:58:40.217509 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 14:58:40.217509 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 14:58:40.217509 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 14:58:40.217509 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 14:58:40.217509 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 14:58:40.217509 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 14:58:40.217509 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 14:58:40.217509 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 14:58:40.217509 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 14:58:40.217509 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Sep 23 14:58:40.217509 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 14:58:40.217509 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Sep 23 14:58:40.217509 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 14:58:40.217509 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 14:58:40.218495 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 14:58:40.218495 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Sep 23 14:58:40.218495 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 14:58:40.218495 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Sep 23 14:58:40.218495 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 14:58:40.218495 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 14:58:40.218495 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 14:58:40.218495 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 14:58:40.218495 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 14:58:40.218495 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 14:58:40.218495 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 23 14:58:40.218495 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Sep 23 14:58:40.218495 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 159/304] 10.222.0.55 () {56 vars in 1099 bytes} [Fri Sep 23 14:58:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 14:58:40.218495 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbb47b5-9a30-49e6-96a9-f10a42ea930f None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:40.307756 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:40.312652 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 146/305] 10.222.0.55 () {60 vars in 1134 bytes} [Fri Sep 23 14:58:40 2022] POST /identity/v3/groups => generated 229 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 14:58:40.320169 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a6e7fa-a4fe-4cb8-b188-839463d87124 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:40.327191 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbb47b5-9a30-49e6-96a9-f10a42ea930f None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:40.377109 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd75814-4aac-473b-a69a-c8be25a4d4d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:40.378221 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd75814-4aac-473b-a69a-c8be25a4d4d0 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:40.378984 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd75814-4aac-473b-a69a-c8be25a4d4d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:40.379442 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd75814-4aac-473b-a69a-c8be25a4d4d0 None admin] PATH_INFO: `/v3/users/71026b5a24794d399d38bbd4bf2eb92b` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:40.397580 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd75814-4aac-473b-a69a-c8be25a4d4d0 None admin] RBAC: Authorizing `identity:get_user(user_id=71026b5a24794d399d38bbd4bf2eb92b)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:40.401807 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd75814-4aac-473b-a69a-c8be25a4d4d0 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:40.404036 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 160/306] 10.222.0.55 () {56 vars in 1177 bytes} [Fri Sep 23 14:58:40 2022] GET /identity/v3/users/71026b5a24794d399d38bbd4bf2eb92b => generated 293 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:58:40.410888 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd75814-4aac-473b-a69a-c8be25a4d4d0 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:40.465212 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a6e7fa-a4fe-4cb8-b188-839463d87124 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:40.479808 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:40.520521 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7b9db2-b09d-492e-9ade-62cbb0e91f46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:40.521380 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7b9db2-b09d-492e-9ade-62cbb0e91f46 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:40.521770 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7b9db2-b09d-492e-9ade-62cbb0e91f46 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:40.522151 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7b9db2-b09d-492e-9ade-62cbb0e91f46 None admin] PATH_INFO: `/v3/roles` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:40.523050 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7b9db2-b09d-492e-9ade-62cbb0e91f46 None admin] RBAC: Adding query filter params (name=member) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:58:40.523452 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7b9db2-b09d-492e-9ade-62cbb0e91f46 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:40.525875 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7b9db2-b09d-492e-9ade-62cbb0e91f46 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:40.537155 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 147/307] 10.222.0.55 () {56 vars in 1101 bytes} [Fri Sep 23 14:58:40 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:58:40.543439 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7b9db2-b09d-492e-9ade-62cbb0e91f46 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:40.549704 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd75814-4aac-473b-a69a-c8be25a4d4d0 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:40.597235 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11882242-698f-4409-8168-d180daa1180c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:40.597776 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11882242-698f-4409-8168-d180daa1180c None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:40.598060 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11882242-698f-4409-8168-d180daa1180c None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:40.598338 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11882242-698f-4409-8168-d180daa1180c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:40.599075 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11882242-698f-4409-8168-d180daa1180c None admin] RBAC: Adding query filter params (role.id=ea002840090d4194bb75fa083a55dc5f, user.id=71026b5a24794d399d38bbd4bf2eb92b, scope.system=all) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:58:40.599391 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11882242-698f-4409-8168-d180daa1180c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:40.600889 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11882242-698f-4409-8168-d180daa1180c None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:40.606588 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 161/308] 10.222.0.55 () {56 vars in 1308 bytes} [Fri Sep 23 14:58:40 2022] GET /identity/v3/role_assignments?user.id=71026b5a24794d399d38bbd4bf2eb92b&scope.system=all&role.id=ea002840090d4194bb75fa083a55dc5f => generated 226 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 14:58:40.672980 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7b9db2-b09d-492e-9ade-62cbb0e91f46 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:40.715613 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e5fcb2-04cd-432a-a09e-60bd65bba97f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:40.716345 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e5fcb2-04cd-432a-a09e-60bd65bba97f None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:40.716625 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e5fcb2-04cd-432a-a09e-60bd65bba97f None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:40.716947 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e5fcb2-04cd-432a-a09e-60bd65bba97f None admin] PATH_INFO: `/v3/users/e501b2f5ce44459c900cbc120eb93931` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:40.718118 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e5fcb2-04cd-432a-a09e-60bd65bba97f None admin] RBAC: Authorizing `identity:get_user(user_id=e501b2f5ce44459c900cbc120eb93931)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:40.719600 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e5fcb2-04cd-432a-a09e-60bd65bba97f None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:40.721649 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 148/309] 10.222.0.55 () {56 vars in 1177 bytes} [Fri Sep 23 14:58:40 2022] GET /identity/v3/users/e501b2f5ce44459c900cbc120eb93931 => generated 293 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 14:58:40.727172 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11882242-698f-4409-8168-d180daa1180c None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:40.738487 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:40.849941 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11882242-698f-4409-8168-d180daa1180c None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:40.892952 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6674c8c0-01b1-4fb6-90d1-3e445034d990 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:40.893730 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6674c8c0-01b1-4fb6-90d1-3e445034d990 None admin] REQUEST_METHOD: `PUT` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:40.894015 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6674c8c0-01b1-4fb6-90d1-3e445034d990 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:40.894292 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6674c8c0-01b1-4fb6-90d1-3e445034d990 None admin] PATH_INFO: `/v3/system/users/e501b2f5ce44459c900cbc120eb93931/roles/af257be2dee54d6b942f58650196582c` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:40.895743 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6674c8c0-01b1-4fb6-90d1-3e445034d990 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=e501b2f5ce44459c900cbc120eb93931, role_id=af257be2dee54d6b942f58650196582c)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:40.897169 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6674c8c0-01b1-4fb6-90d1-3e445034d990 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:40.965076 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64d9494-be98-4e31-91ff-cb391e720c5b None None] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:40.965397 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64d9494-be98-4e31-91ff-cb391e720c5b None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:40.965622 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64d9494-be98-4e31-91ff-cb391e720c5b None None] PATH_INFO: `/` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:40.967089 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 149/310] 10.222.0.55 () {54 vars in 927 bytes} [Fri Sep 23 14:58:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 14:58:40.972022 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f19992-4a19-47e9-9556-96fccf92501d None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:40.972299 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f19992-4a19-47e9-9556-96fccf92501d None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:40.972582 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f19992-4a19-47e9-9556-96fccf92501d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:40.993569 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-f2f19992-4a19-47e9-9556-96fccf92501d None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 14:58:40.995210 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-f2f19992-4a19-47e9-9556-96fccf92501d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 14:58:41.019960 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 162/311] 10.222.0.55 () {58 vars in 1334 bytes} [Fri Sep 23 14:58:40 2022] PUT /identity/v3/system/users/e501b2f5ce44459c900cbc120eb93931/roles/af257be2dee54d6b942f58650196582c => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 14:58:41.035568 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:41.084678 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 150/312] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Sep 23 14:58:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 23 14:58:41.365470 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6674c8c0-01b1-4fb6-90d1-3e445034d990 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:41.439718 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:41.471950 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6674c8c0-01b1-4fb6-90d1-3e445034d990 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:41.507363 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edff104c-462a-4c9b-841f-5e87b9c7ed0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:41.508066 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edff104c-462a-4c9b-841f-5e87b9c7ed0e None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:41.508349 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edff104c-462a-4c9b-841f-5e87b9c7ed0e None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:41.508687 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edff104c-462a-4c9b-841f-5e87b9c7ed0e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:41.519825 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edff104c-462a-4c9b-841f-5e87b9c7ed0e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 14:58:41.525876 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-edff104c-462a-4c9b-841f-5e87b9c7ed0e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 23 14:58:41.525876 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 14:58:41.525876 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 14:58:41.525876 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 14:58:41.525876 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 14:58:41.525876 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 14:58:41.525876 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 14:58:41.525876 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 14:58:41.525876 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 14:58:41.525876 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 14:58:41.525876 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 14:58:41.525876 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 14:58:41.525876 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 14:58:41.525876 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 14:58:41.525876 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 14:58:41.525876 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 14:58:41.526939 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 14:58:41.526939 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 14:58:41.526939 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 14:58:41.526939 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 14:58:41.526939 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 14:58:41.526939 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 14:58:41.526939 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 14:58:41.526939 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 14:58:41.526939 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 14:58:41.526939 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 14:58:41.526939 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 14:58:41.526939 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Sep 23 14:58:41.526939 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 14:58:41.526939 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Sep 23 14:58:41.526939 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 14:58:41.526939 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 14:58:41.527922 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 14:58:41.527922 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Sep 23 14:58:41.527922 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 14:58:41.527922 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Sep 23 14:58:41.527922 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 14:58:41.527922 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 14:58:41.527922 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 14:58:41.527922 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 14:58:41.527922 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 14:58:41.527922 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 14:58:41.527922 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 23 14:58:41.527922 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Sep 23 14:58:41.530044 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 163/313] 10.222.0.55 () {56 vars in 1099 bytes} [Fri Sep 23 14:58:41 2022] GET /identity/v3/roles/reader => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 14:58:41.536054 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f19992-4a19-47e9-9556-96fccf92501d None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:41.642425 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f19992-4a19-47e9-9556-96fccf92501d None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:41.677860 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecc4954-4f9e-4376-afa5-c19cc2207db2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:41.678601 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecc4954-4f9e-4376-afa5-c19cc2207db2 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:41.678758 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecc4954-4f9e-4376-afa5-c19cc2207db2 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:41.678984 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecc4954-4f9e-4376-afa5-c19cc2207db2 None admin] PATH_INFO: `/v3/roles` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:41.679641 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecc4954-4f9e-4376-afa5-c19cc2207db2 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:58:41.679937 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecc4954-4f9e-4376-afa5-c19cc2207db2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:41.681423 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecc4954-4f9e-4376-afa5-c19cc2207db2 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:41.690147 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 151/314] 10.222.0.55 () {56 vars in 1101 bytes} [Fri Sep 23 14:58:41 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:58:41.695837 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edff104c-462a-4c9b-841f-5e87b9c7ed0e None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:41.803871 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edff104c-462a-4c9b-841f-5e87b9c7ed0e None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:41.833920 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:41.840381 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d73e90-fd04-46e8-867b-0c8e570508e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:41.841028 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d73e90-fd04-46e8-867b-0c8e570508e1 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:41.841298 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d73e90-fd04-46e8-867b-0c8e570508e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:41.841576 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d73e90-fd04-46e8-867b-0c8e570508e1 None admin] PATH_INFO: `/v3/users/3a099085ae5b4b5780b0f3c4d86b19f0` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:41.842704 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d73e90-fd04-46e8-867b-0c8e570508e1 None admin] RBAC: Authorizing `identity:get_user(user_id=3a099085ae5b4b5780b0f3c4d86b19f0)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:41.844153 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d73e90-fd04-46e8-867b-0c8e570508e1 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:41.845912 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 164/315] 10.222.0.55 () {56 vars in 1177 bytes} [Fri Sep 23 14:58:41 2022] GET /identity/v3/users/3a099085ae5b4b5780b0f3c4d86b19f0 => generated 297 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:58:41.852162 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecc4954-4f9e-4376-afa5-c19cc2207db2 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:41.886348 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:41.962634 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecc4954-4f9e-4376-afa5-c19cc2207db2 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:41.999247 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dddb90f-04ae-4552-9226-25fdfbb8709a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:41.999863 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dddb90f-04ae-4552-9226-25fdfbb8709a None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:42.000180 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dddb90f-04ae-4552-9226-25fdfbb8709a None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:42.000451 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dddb90f-04ae-4552-9226-25fdfbb8709a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:42.073755 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dddb90f-04ae-4552-9226-25fdfbb8709a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:42.075209 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dddb90f-04ae-4552-9226-25fdfbb8709a None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:42.106198 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:42.178775 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 152/316] 10.222.0.55 () {58 vars in 1282 bytes} [Fri Sep 23 14:58:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 23 14:58:42.184638 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d73e90-fd04-46e8-867b-0c8e570508e1 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:42.293018 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d73e90-fd04-46e8-867b-0c8e570508e1 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:42.329214 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64efc249-6b9a-4b5e-b956-ea41c27e604f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:42.329853 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64efc249-6b9a-4b5e-b956-ea41c27e604f None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:42.330138 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64efc249-6b9a-4b5e-b956-ea41c27e604f None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:42.330408 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64efc249-6b9a-4b5e-b956-ea41c27e604f None admin] PATH_INFO: `/v3/projects/346756b911774d55af2a9ad3de2143b4` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:42.331522 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64efc249-6b9a-4b5e-b956-ea41c27e604f None admin] RBAC: Authorizing `identity:get_project(project_id=346756b911774d55af2a9ad3de2143b4)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:42.332977 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64efc249-6b9a-4b5e-b956-ea41c27e604f None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:42.334692 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 165/317] 10.222.0.55 () {56 vars in 1186 bytes} [Fri Sep 23 14:58:42 2022] GET /identity/v3/projects/346756b911774d55af2a9ad3de2143b4 => generated 302 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:58:42.341359 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dddb90f-04ae-4552-9226-25fdfbb8709a None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:42.445369 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dddb90f-04ae-4552-9226-25fdfbb8709a None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:42.476230 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:42.482399 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39809689-7b7c-4fea-b6c3-390d23cc8d2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:42.483010 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39809689-7b7c-4fea-b6c3-390d23cc8d2a None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:42.483285 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39809689-7b7c-4fea-b6c3-390d23cc8d2a None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:42.483557 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39809689-7b7c-4fea-b6c3-390d23cc8d2a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:42.484311 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39809689-7b7c-4fea-b6c3-390d23cc8d2a None admin] RBAC: Adding query filter params (role.id=ea002840090d4194bb75fa083a55dc5f, scope.project.id=346756b911774d55af2a9ad3de2143b4, user.id=3a099085ae5b4b5780b0f3c4d86b19f0) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:58:42.484614 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39809689-7b7c-4fea-b6c3-390d23cc8d2a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:42.486243 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39809689-7b7c-4fea-b6c3-390d23cc8d2a None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:42.492476 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 153/318] 10.222.0.55 () {56 vars in 1374 bytes} [Fri Sep 23 14:58:42 2022] GET /identity/v3/role_assignments?user.id=3a099085ae5b4b5780b0f3c4d86b19f0&scope.project.id=346756b911774d55af2a9ad3de2143b4&role.id=ea002840090d4194bb75fa083a55dc5f => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:58:42.685986 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3ba9b0-5fa1-459a-91f6-a9c74608a32c None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:42.686338 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3ba9b0-5fa1-459a-91f6-a9c74608a32c None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:42.686515 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3ba9b0-5fa1-459a-91f6-a9c74608a32c None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:42.688119 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 166/319] 10.222.0.55 () {54 vars in 927 bytes} [Fri Sep 23 14:58:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 14:58:42.693540 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b4ca5e-646e-43f4-a2a4-a0b7fbfae13c None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:42.693994 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b4ca5e-646e-43f4-a2a4-a0b7fbfae13c None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:42.694098 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b4ca5e-646e-43f4-a2a4-a0b7fbfae13c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:42.714394 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-14b4ca5e-646e-43f4-a2a4-a0b7fbfae13c None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 14:58:42.715616 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-14b4ca5e-646e-43f4-a2a4-a0b7fbfae13c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 14:58:42.795221 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 154/320] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Sep 23 14:58:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 14:58:42.948660 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d64b8f-ff25-4943-a2f3-a53c9afa8109 None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:42.949164 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d64b8f-ff25-4943-a2f3-a53c9afa8109 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:42.949164 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d64b8f-ff25-4943-a2f3-a53c9afa8109 None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:42.950565 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 167/321] 10.222.0.55 () {54 vars in 927 bytes} [Fri Sep 23 14:58:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 14:58:42.955546 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b7b473-1b0f-495a-9dc3-abd9a245006a None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:42.955830 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b7b473-1b0f-495a-9dc3-abd9a245006a None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:42.957163 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b7b473-1b0f-495a-9dc3-abd9a245006a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:42.974610 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-f5b7b473-1b0f-495a-9dc3-abd9a245006a None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 14:58:42.975780 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-f5b7b473-1b0f-495a-9dc3-abd9a245006a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 14:58:43.045789 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 155/322] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Sep 23 14:58:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 14:58:43.068218 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d64b8f-ff25-4943-a2f3-a53c9afa8109 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:43.143129 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:43.183724 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d64b8f-ff25-4943-a2f3-a53c9afa8109 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:43.224825 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72618f57-1ab5-417a-b1c6-8564c4ebc908 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:43.225472 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72618f57-1ab5-417a-b1c6-8564c4ebc908 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:43.225758 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72618f57-1ab5-417a-b1c6-8564c4ebc908 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:43.226035 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72618f57-1ab5-417a-b1c6-8564c4ebc908 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:43.227138 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72618f57-1ab5-417a-b1c6-8564c4ebc908 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:43.228647 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72618f57-1ab5-417a-b1c6-8564c4ebc908 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:43.230380 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 168/323] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Sep 23 14:58:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:58:43.235855 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b7b473-1b0f-495a-9dc3-abd9a245006a None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:43.271787 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63887d85-f277-492c-aa95-b78d12dfa1a6 None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:43.272128 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63887d85-f277-492c-aa95-b78d12dfa1a6 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:43.272406 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63887d85-f277-492c-aa95-b78d12dfa1a6 None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:43.273880 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 169/324] 10.222.0.55 () {54 vars in 927 bytes} [Fri Sep 23 14:58:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 14:58:43.279038 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601f69d3-6fa8-4f5a-a57b-12f6c883395a None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:43.279038 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601f69d3-6fa8-4f5a-a57b-12f6c883395a None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:43.279329 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601f69d3-6fa8-4f5a-a57b-12f6c883395a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:43.295941 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:43.297779 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-601f69d3-6fa8-4f5a-a57b-12f6c883395a None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 14:58:43.298983 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-601f69d3-6fa8-4f5a-a57b-12f6c883395a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 14:58:43.345694 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b7b473-1b0f-495a-9dc3-abd9a245006a None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:43.371054 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 170/325] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Sep 23 14:58:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 23 14:58:43.372168 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601f69d3-6fa8-4f5a-a57b-12f6c883395a None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:43.379828 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57122c5-3590-4640-aa26-10e9e164c691 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:43.380468 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57122c5-3590-4640-aa26-10e9e164c691 None admin] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:43.380741 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57122c5-3590-4640-aa26-10e9e164c691 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:43.381007 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57122c5-3590-4640-aa26-10e9e164c691 None admin] PATH_INFO: `/v3/groups` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:43.381647 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57122c5-3590-4640-aa26-10e9e164c691 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:43.383061 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57122c5-3590-4640-aa26-10e9e164c691 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:43.442747 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:43.464869 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 156/326] 10.222.0.55 () {60 vars in 1134 bytes} [Fri Sep 23 14:58:43 2022] POST /identity/v3/groups => generated 226 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 14:58:43.484494 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601f69d3-6fa8-4f5a-a57b-12f6c883395a None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:43.524634 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3674db0-30fa-4af7-9463-00f09b0cd5b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:43.525296 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3674db0-30fa-4af7-9463-00f09b0cd5b6 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:43.525584 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3674db0-30fa-4af7-9463-00f09b0cd5b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:43.525860 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3674db0-30fa-4af7-9463-00f09b0cd5b6 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:43.538370 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3674db0-30fa-4af7-9463-00f09b0cd5b6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 14:58:43.543731 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-a3674db0-30fa-4af7-9463-00f09b0cd5b6 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 23 14:58:43.543731 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 14:58:43.543731 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 14:58:43.543731 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 14:58:43.543731 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 14:58:43.543731 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 14:58:43.543731 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 14:58:43.543731 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 14:58:43.543731 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 14:58:43.543731 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 14:58:43.543731 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 14:58:43.543731 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 14:58:43.543731 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 14:58:43.543731 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 14:58:43.543731 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 14:58:43.543731 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 14:58:43.544930 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 14:58:43.544930 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 14:58:43.544930 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 14:58:43.544930 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 14:58:43.544930 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 14:58:43.544930 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 14:58:43.544930 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 14:58:43.544930 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 14:58:43.544930 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 14:58:43.544930 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 14:58:43.544930 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 14:58:43.544930 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Sep 23 14:58:43.544930 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 14:58:43.544930 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Sep 23 14:58:43.544930 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 14:58:43.544930 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 14:58:43.545748 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 14:58:43.545748 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Sep 23 14:58:43.545748 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 14:58:43.545748 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Sep 23 14:58:43.545748 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 14:58:43.545748 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 14:58:43.545748 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 14:58:43.545748 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 14:58:43.545748 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 14:58:43.545748 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 14:58:43.545748 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 23 14:58:43.545748 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Sep 23 14:58:43.545748 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 171/327] 10.222.0.55 () {56 vars in 1099 bytes} [Fri Sep 23 14:58:43 2022] GET /identity/v3/roles/reader => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 14:58:43.552830 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57122c5-3590-4640-aa26-10e9e164c691 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:43.639418 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3674db0-30fa-4af7-9463-00f09b0cd5b6 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:43.673961 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57122c5-3590-4640-aa26-10e9e164c691 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:43.690125 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:43.715175 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f90cfe9-e3d9-423a-af3e-2a67ebefd5a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:43.715791 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f90cfe9-e3d9-423a-af3e-2a67ebefd5a4 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:43.716086 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f90cfe9-e3d9-423a-af3e-2a67ebefd5a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:43.716380 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f90cfe9-e3d9-423a-af3e-2a67ebefd5a4 None admin] PATH_INFO: `/v3/roles` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:43.717439 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f90cfe9-e3d9-423a-af3e-2a67ebefd5a4 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:58:43.717439 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f90cfe9-e3d9-423a-af3e-2a67ebefd5a4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:43.718676 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f90cfe9-e3d9-423a-af3e-2a67ebefd5a4 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:43.728766 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 157/328] 10.222.0.55 () {56 vars in 1101 bytes} [Fri Sep 23 14:58:43 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:58:43.734469 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f90cfe9-e3d9-423a-af3e-2a67ebefd5a4 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:43.767596 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3674db0-30fa-4af7-9463-00f09b0cd5b6 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:43.795394 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:43.809172 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba532d78-7ae4-4525-8cf6-dddf39431fa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:43.809817 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba532d78-7ae4-4525-8cf6-dddf39431fa8 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:43.810095 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba532d78-7ae4-4525-8cf6-dddf39431fa8 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:43.810367 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba532d78-7ae4-4525-8cf6-dddf39431fa8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:43.822531 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba532d78-7ae4-4525-8cf6-dddf39431fa8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 14:58:43.828503 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-ba532d78-7ae4-4525-8cf6-dddf39431fa8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 23 14:58:43.828503 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 14:58:43.828503 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 14:58:43.828503 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 14:58:43.828503 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 14:58:43.828503 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 14:58:43.828503 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 14:58:43.828503 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 14:58:43.828503 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 14:58:43.828503 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 14:58:43.828503 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 14:58:43.828503 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 14:58:43.828503 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 14:58:43.828503 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 14:58:43.828503 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 14:58:43.828503 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 14:58:43.829706 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 14:58:43.829706 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 14:58:43.829706 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 14:58:43.829706 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 14:58:43.829706 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 14:58:43.829706 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 14:58:43.829706 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 14:58:43.829706 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 14:58:43.829706 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 14:58:43.829706 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 14:58:43.829706 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 14:58:43.829706 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Sep 23 14:58:43.829706 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 14:58:43.829706 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Sep 23 14:58:43.829706 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 14:58:43.829706 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 14:58:43.830942 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 14:58:43.830942 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Sep 23 14:58:43.830942 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 14:58:43.830942 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Sep 23 14:58:43.830942 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 14:58:43.830942 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 14:58:43.830942 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 14:58:43.830942 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 14:58:43.830942 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 14:58:43.830942 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 14:58:43.830942 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 23 14:58:43.830942 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Sep 23 14:58:43.830942 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 172/329] 10.222.0.55 () {56 vars in 1099 bytes} [Fri Sep 23 14:58:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 23 14:58:43.836216 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba532d78-7ae4-4525-8cf6-dddf39431fa8 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:43.867470 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f90cfe9-e3d9-423a-af3e-2a67ebefd5a4 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:43.944670 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5c5914-1363-4761-bdcb-31422120605b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:43.945412 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5c5914-1363-4761-bdcb-31422120605b None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:43.945701 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5c5914-1363-4761-bdcb-31422120605b None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:43.945975 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5c5914-1363-4761-bdcb-31422120605b None admin] PATH_INFO: `/v3/users/71026b5a24794d399d38bbd4bf2eb92b` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:43.948143 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5c5914-1363-4761-bdcb-31422120605b None admin] RBAC: Authorizing `identity:get_user(user_id=71026b5a24794d399d38bbd4bf2eb92b)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:43.949692 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5c5914-1363-4761-bdcb-31422120605b None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:43.951673 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 158/330] 10.222.0.55 () {56 vars in 1177 bytes} [Fri Sep 23 14:58:43 2022] GET /identity/v3/users/71026b5a24794d399d38bbd4bf2eb92b => generated 293 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 14:58:43.957432 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5c5914-1363-4761-bdcb-31422120605b None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:43.966079 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba532d78-7ae4-4525-8cf6-dddf39431fa8 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:44.011668 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9236397c-e610-4fc5-a464-b735e8857dd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:44.012744 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9236397c-e610-4fc5-a464-b735e8857dd1 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:44.013200 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9236397c-e610-4fc5-a464-b735e8857dd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:44.013668 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9236397c-e610-4fc5-a464-b735e8857dd1 None admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:44.014519 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9236397c-e610-4fc5-a464-b735e8857dd1 None admin] RBAC: Adding query filter params (name=member) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:58:44.016263 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9236397c-e610-4fc5-a464-b735e8857dd1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:44.018508 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9236397c-e610-4fc5-a464-b735e8857dd1 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:44.021993 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:44.033108 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 173/331] 10.222.0.55 () {56 vars in 1101 bytes} [Fri Sep 23 14:58:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 14:58:44.039068 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9236397c-e610-4fc5-a464-b735e8857dd1 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:44.095395 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5c5914-1363-4761-bdcb-31422120605b None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:44.107201 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:44.144855 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c2051a-3f5b-4191-bbf4-325d7f42f2b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:44.145608 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c2051a-3f5b-4191-bbf4-325d7f42f2b2 None admin] REQUEST_METHOD: `PUT` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:44.145972 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c2051a-3f5b-4191-bbf4-325d7f42f2b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:44.146312 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c2051a-3f5b-4191-bbf4-325d7f42f2b2 None admin] PATH_INFO: `/v3/system/users/71026b5a24794d399d38bbd4bf2eb92b/roles/ea002840090d4194bb75fa083a55dc5f` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:44.148335 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c2051a-3f5b-4191-bbf4-325d7f42f2b2 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=71026b5a24794d399d38bbd4bf2eb92b, role_id=ea002840090d4194bb75fa083a55dc5f)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:44.149931 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c2051a-3f5b-4191-bbf4-325d7f42f2b2 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:44.175797 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9236397c-e610-4fc5-a464-b735e8857dd1 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:44.213820 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a21e63-1046-40dc-8a24-4f3f128d56e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:44.214458 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a21e63-1046-40dc-8a24-4f3f128d56e6 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:44.214743 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a21e63-1046-40dc-8a24-4f3f128d56e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:44.215022 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a21e63-1046-40dc-8a24-4f3f128d56e6 None admin] PATH_INFO: `/v3/users/e501b2f5ce44459c900cbc120eb93931` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:44.216181 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a21e63-1046-40dc-8a24-4f3f128d56e6 None admin] RBAC: Authorizing `identity:get_user(user_id=e501b2f5ce44459c900cbc120eb93931)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:44.217672 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a21e63-1046-40dc-8a24-4f3f128d56e6 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:44.219252 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 174/332] 10.222.0.55 () {56 vars in 1177 bytes} [Fri Sep 23 14:58:44 2022] GET /identity/v3/users/e501b2f5ce44459c900cbc120eb93931 => generated 293 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:58:44.225890 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a21e63-1046-40dc-8a24-4f3f128d56e6 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:44.300296 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 159/333] 10.222.0.55 () {58 vars in 1334 bytes} [Fri Sep 23 14:58:43 2022] PUT /identity/v3/system/users/71026b5a24794d399d38bbd4bf2eb92b/roles/ea002840090d4194bb75fa083a55dc5f => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 14:58:44.355168 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a21e63-1046-40dc-8a24-4f3f128d56e6 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:44.372493 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:44.402126 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4915ad11-66ac-422b-b9c7-aea4557edff1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:44.403004 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4915ad11-66ac-422b-b9c7-aea4557edff1 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:44.403482 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4915ad11-66ac-422b-b9c7-aea4557edff1 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:44.403925 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4915ad11-66ac-422b-b9c7-aea4557edff1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:44.404956 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4915ad11-66ac-422b-b9c7-aea4557edff1 None admin] RBAC: Adding query filter params (role.id=af257be2dee54d6b942f58650196582c, user.id=e501b2f5ce44459c900cbc120eb93931, scope.system=all) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:58:44.405411 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4915ad11-66ac-422b-b9c7-aea4557edff1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:44.407335 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4915ad11-66ac-422b-b9c7-aea4557edff1 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:44.423554 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 175/334] 10.222.0.55 () {56 vars in 1308 bytes} [Fri Sep 23 14:58:44 2022] GET /identity/v3/role_assignments?user.id=e501b2f5ce44459c900cbc120eb93931&scope.system=all&role.id=af257be2dee54d6b942f58650196582c => generated 510 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:58:46.778234 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb9c13a-c4ee-4aac-b299-d41aa713094b None None] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:46.779031 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb9c13a-c4ee-4aac-b299-d41aa713094b None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:46.779031 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb9c13a-c4ee-4aac-b299-d41aa713094b None None] PATH_INFO: `/` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:46.780360 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 160/335] 10.222.0.55 () {54 vars in 927 bytes} [Fri Sep 23 14:58:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 14:58:46.787232 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8da9671-e073-4001-947c-0cbda964f80f None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:46.788588 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8da9671-e073-4001-947c-0cbda964f80f None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:46.789369 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8da9671-e073-4001-947c-0cbda964f80f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:46.828379 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-b8da9671-e073-4001-947c-0cbda964f80f None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 14:58:46.830921 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-b8da9671-e073-4001-947c-0cbda964f80f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 14:58:46.859607 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e86e18-809f-4ce3-985d-6b74a5306b8a None None] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:46.860022 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e86e18-809f-4ce3-985d-6b74a5306b8a None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:46.860176 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e86e18-809f-4ce3-985d-6b74a5306b8a None None] PATH_INFO: `/` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:46.863516 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 161/336] 10.222.0.55 () {54 vars in 927 bytes} [Fri Sep 23 14:58:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 14:58:46.868264 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a439873-d604-4dad-a4ab-8d590621b7e5 None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:46.868556 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a439873-d604-4dad-a4ab-8d590621b7e5 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:46.868816 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a439873-d604-4dad-a4ab-8d590621b7e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:46.899787 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-7a439873-d604-4dad-a4ab-8d590621b7e5 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 14:58:46.901018 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-7a439873-d604-4dad-a4ab-8d590621b7e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 14:58:46.975314 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 176/337] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Sep 23 14:58:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 14:58:47.037276 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 162/338] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Sep 23 14:58:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 14:58:47.465414 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8da9671-e073-4001-947c-0cbda964f80f None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:47.529498 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337af734-5b3d-4cec-bf0c-8764d08bd1ca None None] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:47.529765 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337af734-5b3d-4cec-bf0c-8764d08bd1ca None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:47.530163 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337af734-5b3d-4cec-bf0c-8764d08bd1ca None None] PATH_INFO: `/` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:47.531754 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 163/339] 10.222.0.55 () {54 vars in 927 bytes} [Fri Sep 23 14:58:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 14:58:47.537206 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07be1ef6-f7c1-448e-9d66-feaee32a8a39 None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:47.537507 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07be1ef6-f7c1-448e-9d66-feaee32a8a39 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:47.537741 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07be1ef6-f7c1-448e-9d66-feaee32a8a39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:47.565395 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-07be1ef6-f7c1-448e-9d66-feaee32a8a39 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 14:58:47.566605 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-07be1ef6-f7c1-448e-9d66-feaee32a8a39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 14:58:47.641649 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:47.650078 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8da9671-e073-4001-947c-0cbda964f80f None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:47.657947 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:47.709761 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba50683-cd97-4ca3-ac8a-a55d5f8e6256 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:47.710182 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba50683-cd97-4ca3-ac8a-a55d5f8e6256 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:47.710470 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba50683-cd97-4ca3-ac8a-a55d5f8e6256 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:47.710744 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba50683-cd97-4ca3-ac8a-a55d5f8e6256 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:47.727726 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 164/340] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Sep 23 14:58:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 14:58:47.733126 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba50683-cd97-4ca3-ac8a-a55d5f8e6256 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 14:58:47.739275 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-7ba50683-cd97-4ca3-ac8a-a55d5f8e6256 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 23 14:58:47.739275 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 14:58:47.739275 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 14:58:47.739275 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 14:58:47.739275 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 14:58:47.739275 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 14:58:47.739275 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 14:58:47.739275 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 14:58:47.739275 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 14:58:47.739275 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 14:58:47.739275 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 14:58:47.739275 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 14:58:47.739275 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 14:58:47.739275 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 14:58:47.739275 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 14:58:47.739275 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 14:58:47.741006 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 14:58:47.741006 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 14:58:47.741006 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 14:58:47.741006 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 14:58:47.741006 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 14:58:47.741006 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 14:58:47.741006 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 14:58:47.741006 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 14:58:47.741006 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 14:58:47.741006 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 14:58:47.741006 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 14:58:47.741006 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Sep 23 14:58:47.741006 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 14:58:47.741006 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Sep 23 14:58:47.741006 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 14:58:47.741006 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 14:58:47.742571 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 14:58:47.742571 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Sep 23 14:58:47.742571 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 14:58:47.742571 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Sep 23 14:58:47.742571 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 14:58:47.742571 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 14:58:47.742571 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 14:58:47.742571 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 14:58:47.742571 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 14:58:47.742571 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 14:58:47.742571 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 23 14:58:47.742571 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Sep 23 14:58:47.742571 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 177/341] 10.222.0.55 () {56 vars in 1099 bytes} [Fri Sep 23 14:58:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 14:58:47.748813 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07be1ef6-f7c1-448e-9d66-feaee32a8a39 None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:47.850167 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba50683-cd97-4ca3-ac8a-a55d5f8e6256 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:47.908571 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07be1ef6-f7c1-448e-9d66-feaee32a8a39 None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:47.969617 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd17eec-51b9-4293-92cc-876450ea95ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:47.970252 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd17eec-51b9-4293-92cc-876450ea95ca None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:47.970689 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd17eec-51b9-4293-92cc-876450ea95ca None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:47.970971 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd17eec-51b9-4293-92cc-876450ea95ca None admin] PATH_INFO: `/v3/roles` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:47.971651 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd17eec-51b9-4293-92cc-876450ea95ca None admin] RBAC: Adding query filter params (name=member) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:58:47.971957 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd17eec-51b9-4293-92cc-876450ea95ca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:47.973361 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd17eec-51b9-4293-92cc-876450ea95ca None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:47.991332 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 165/342] 10.222.0.55 () {56 vars in 1101 bytes} [Fri Sep 23 14:58:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:58:47.998397 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd17eec-51b9-4293-92cc-876450ea95ca None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:48.047300 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:48.117931 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba50683-cd97-4ca3-ac8a-a55d5f8e6256 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:48.173575 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ae0280-0969-4e6c-ba5f-23199bc8c4d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:48.174218 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ae0280-0969-4e6c-ba5f-23199bc8c4d6 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:48.174491 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ae0280-0969-4e6c-ba5f-23199bc8c4d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:48.174794 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ae0280-0969-4e6c-ba5f-23199bc8c4d6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:48.182201 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd17eec-51b9-4293-92cc-876450ea95ca None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:48.190047 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ae0280-0969-4e6c-ba5f-23199bc8c4d6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 14:58:48.195314 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-f6ae0280-0969-4e6c-ba5f-23199bc8c4d6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 23 14:58:48.195314 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 14:58:48.195314 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 14:58:48.195314 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 14:58:48.195314 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 14:58:48.195314 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 14:58:48.195314 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 14:58:48.195314 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 14:58:48.195314 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 14:58:48.195314 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 14:58:48.195314 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 14:58:48.195314 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 14:58:48.195314 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 14:58:48.195314 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 14:58:48.195314 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 14:58:48.195314 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 14:58:48.197607 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 14:58:48.197607 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 14:58:48.197607 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 14:58:48.197607 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 14:58:48.197607 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 14:58:48.197607 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 14:58:48.197607 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 14:58:48.197607 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 14:58:48.197607 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 14:58:48.197607 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 14:58:48.197607 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 14:58:48.197607 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Sep 23 14:58:48.197607 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 14:58:48.197607 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Sep 23 14:58:48.197607 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 14:58:48.197607 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 14:58:48.199141 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 14:58:48.199141 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Sep 23 14:58:48.199141 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 14:58:48.199141 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Sep 23 14:58:48.199141 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 14:58:48.199141 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 14:58:48.199141 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 14:58:48.199141 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 14:58:48.199141 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 14:58:48.199141 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 14:58:48.199141 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 23 14:58:48.199141 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Sep 23 14:58:48.199141 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 178/343] 10.222.0.55 () {56 vars in 1099 bytes} [Fri Sep 23 14:58:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 348 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 14:58:48.204103 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ae0280-0969-4e6c-ba5f-23199bc8c4d6 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:48.256594 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:48.257925 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0115a5b1-a5ae-4d07-ad0f-fc988a895147 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:48.259644 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0115a5b1-a5ae-4d07-ad0f-fc988a895147 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:48.260164 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0115a5b1-a5ae-4d07-ad0f-fc988a895147 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:48.260778 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0115a5b1-a5ae-4d07-ad0f-fc988a895147 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:48.373580 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0115a5b1-a5ae-4d07-ad0f-fc988a895147 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:48.375636 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0115a5b1-a5ae-4d07-ad0f-fc988a895147 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:48.380851 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ae0280-0969-4e6c-ba5f-23199bc8c4d6 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:48.412288 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:48.429769 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad3e0a3-2004-49a6-8816-1bdddee00383 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:48.430418 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad3e0a3-2004-49a6-8816-1bdddee00383 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:48.431758 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad3e0a3-2004-49a6-8816-1bdddee00383 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:48.432266 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad3e0a3-2004-49a6-8816-1bdddee00383 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:48.447220 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad3e0a3-2004-49a6-8816-1bdddee00383 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 14:58:48.454050 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-9ad3e0a3-2004-49a6-8816-1bdddee00383 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 23 14:58:48.454050 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 14:58:48.454050 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 14:58:48.454050 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 14:58:48.454050 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 14:58:48.454050 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 14:58:48.454050 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 14:58:48.454050 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 14:58:48.454050 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 14:58:48.454050 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 14:58:48.454050 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 14:58:48.454050 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 14:58:48.454050 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 14:58:48.454050 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 14:58:48.454050 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 14:58:48.454050 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 14:58:48.455012 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 14:58:48.455012 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 14:58:48.455012 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 14:58:48.455012 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 14:58:48.455012 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 14:58:48.455012 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 14:58:48.455012 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 14:58:48.455012 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 14:58:48.455012 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 14:58:48.455012 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 14:58:48.455012 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 14:58:48.455012 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Sep 23 14:58:48.455012 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 14:58:48.455012 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Sep 23 14:58:48.455012 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 14:58:48.455012 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 14:58:48.455825 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 14:58:48.455825 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Sep 23 14:58:48.455825 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 14:58:48.455825 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Sep 23 14:58:48.455825 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 14:58:48.455825 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 14:58:48.455825 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 14:58:48.455825 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 14:58:48.455825 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 14:58:48.455825 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 14:58:48.455825 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 23 14:58:48.455825 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Sep 23 14:58:48.455825 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 179/344] 10.222.0.55 () {56 vars in 1114 bytes} [Fri Sep 23 14:58:48 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 14:58:48.457061 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad3e0a3-2004-49a6-8816-1bdddee00383 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:48.509704 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 166/345] 10.222.0.55 () {58 vars in 1282 bytes} [Fri Sep 23 14:58:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 512 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 23 14:58:48.510934 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e67add-de91-4edc-b512-e795fc2bcb7d None None] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:48.511218 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e67add-de91-4edc-b512-e795fc2bcb7d None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:48.517530 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e67add-de91-4edc-b512-e795fc2bcb7d None None] PATH_INFO: `/` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:48.519318 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 167/346] 10.222.0.55 () {54 vars in 927 bytes} [Fri Sep 23 14:58:48 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 14:58:48.520620 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e67add-de91-4edc-b512-e795fc2bcb7d None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:48.585109 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:48.616103 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad3e0a3-2004-49a6-8816-1bdddee00383 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:48.660184 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edc0aa7-8a9b-4241-80cd-8c2410e888aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:48.660959 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edc0aa7-8a9b-4241-80cd-8c2410e888aa None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:48.661331 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edc0aa7-8a9b-4241-80cd-8c2410e888aa None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:48.661709 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edc0aa7-8a9b-4241-80cd-8c2410e888aa None admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:48.662535 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edc0aa7-8a9b-4241-80cd-8c2410e888aa None admin] RBAC: Adding query filter params (name=member) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:58:48.662983 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edc0aa7-8a9b-4241-80cd-8c2410e888aa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:48.665868 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e67add-de91-4edc-b512-e795fc2bcb7d None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:48.667446 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edc0aa7-8a9b-4241-80cd-8c2410e888aa None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:48.678079 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 180/347] 10.222.0.55 () {56 vars in 1101 bytes} [Fri Sep 23 14:58:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:58:48.678966 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edc0aa7-8a9b-4241-80cd-8c2410e888aa None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:48.704855 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:48.713477 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e89cf49-861a-4585-8b2d-5f9a27317d4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:48.714635 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e89cf49-861a-4585-8b2d-5f9a27317d4e None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:48.715056 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e89cf49-861a-4585-8b2d-5f9a27317d4e None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:48.715457 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e89cf49-861a-4585-8b2d-5f9a27317d4e None admin] PATH_INFO: `/v3/roles` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:48.716294 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e89cf49-861a-4585-8b2d-5f9a27317d4e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:58:48.716766 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e89cf49-861a-4585-8b2d-5f9a27317d4e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:48.718994 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e89cf49-861a-4585-8b2d-5f9a27317d4e None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:48.734914 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 168/348] 10.222.0.55 () {56 vars in 1111 bytes} [Fri Sep 23 14:58:48 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:58:48.736802 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec76c1c9-a995-42e7-8ca6-98c100483fc4 None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:48.737225 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec76c1c9-a995-42e7-8ca6-98c100483fc4 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:48.745028 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec76c1c9-a995-42e7-8ca6-98c100483fc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:48.762953 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-ec76c1c9-a995-42e7-8ca6-98c100483fc4 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 14:58:48.764439 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-ec76c1c9-a995-42e7-8ca6-98c100483fc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 14:58:48.823846 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edc0aa7-8a9b-4241-80cd-8c2410e888aa None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:48.872495 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520f0a6f-baaf-4fce-aaf3-78e5f43bc772 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:48.874787 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520f0a6f-baaf-4fce-aaf3-78e5f43bc772 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:48.875856 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520f0a6f-baaf-4fce-aaf3-78e5f43bc772 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:48.876030 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 169/349] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Sep 23 14:58:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 14:58:48.876949 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520f0a6f-baaf-4fce-aaf3-78e5f43bc772 None admin] PATH_INFO: `/v3/projects/346756b911774d55af2a9ad3de2143b4` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:48.877067 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec76c1c9-a995-42e7-8ca6-98c100483fc4 None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:48.878903 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520f0a6f-baaf-4fce-aaf3-78e5f43bc772 None admin] RBAC: Authorizing `identity:get_project(project_id=346756b911774d55af2a9ad3de2143b4)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:48.882866 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520f0a6f-baaf-4fce-aaf3-78e5f43bc772 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:48.885376 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 181/350] 10.222.0.55 () {56 vars in 1186 bytes} [Fri Sep 23 14:58:48 2022] GET /identity/v3/projects/346756b911774d55af2a9ad3de2143b4 => generated 302 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:58:48.886477 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520f0a6f-baaf-4fce-aaf3-78e5f43bc772 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:48.906575 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:49.028332 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520f0a6f-baaf-4fce-aaf3-78e5f43bc772 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:49.047020 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec76c1c9-a995-42e7-8ca6-98c100483fc4 None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:49.076828 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:49.093971 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69549d4b-5278-4578-85e1-50684ce8ccf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:49.095493 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69549d4b-5278-4578-85e1-50684ce8ccf5 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:49.096357 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4107f1-7bcc-40bb-a41d-e108bae0829e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:49.097232 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4107f1-7bcc-40bb-a41d-e108bae0829e None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:49.097764 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69549d4b-5278-4578-85e1-50684ce8ccf5 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:49.098218 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69549d4b-5278-4578-85e1-50684ce8ccf5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:49.098689 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4107f1-7bcc-40bb-a41d-e108bae0829e None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:49.099625 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4107f1-7bcc-40bb-a41d-e108bae0829e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:49.240719 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69549d4b-5278-4578-85e1-50684ce8ccf5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:49.241254 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4107f1-7bcc-40bb-a41d-e108bae0829e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:49.242205 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69549d4b-5278-4578-85e1-50684ce8ccf5 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:49.242968 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4107f1-7bcc-40bb-a41d-e108bae0829e None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:49.265226 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:49.377063 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 170/351] 10.222.0.55 () {58 vars in 1282 bytes} [Fri Sep 23 14:58:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 501 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 23 14:58:49.378278 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69549d4b-5278-4578-85e1-50684ce8ccf5 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:49.384087 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 182/352] 10.222.0.55 () {58 vars in 1282 bytes} [Fri Sep 23 14:58:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 498 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 23 14:58:49.385028 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4107f1-7bcc-40bb-a41d-e108bae0829e None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:49.421140 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:49.511305 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4107f1-7bcc-40bb-a41d-e108bae0829e None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:49.513283 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69549d4b-5278-4578-85e1-50684ce8ccf5 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:49.529181 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:49.557783 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882e7ffe-25d9-415c-bd16-384389c5d3bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:49.558429 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882e7ffe-25d9-415c-bd16-384389c5d3bb None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:49.558742 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882e7ffe-25d9-415c-bd16-384389c5d3bb None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:49.559015 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882e7ffe-25d9-415c-bd16-384389c5d3bb None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:49.560695 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8002a3-344e-4aa9-bc91-4cc837cacb28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:49.561418 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8002a3-344e-4aa9-bc91-4cc837cacb28 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:49.561827 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8002a3-344e-4aa9-bc91-4cc837cacb28 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:49.562176 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8002a3-344e-4aa9-bc91-4cc837cacb28 None admin] PATH_INFO: `/v3/groups/9980b9236dfe4cdabb8317a18b888217` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:49.569044 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8002a3-344e-4aa9-bc91-4cc837cacb28 None admin] RBAC: Authorizing `identity:get_group(group_id=9980b9236dfe4cdabb8317a18b888217)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:49.570527 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8002a3-344e-4aa9-bc91-4cc837cacb28 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:49.572489 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882e7ffe-25d9-415c-bd16-384389c5d3bb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 14:58:49.573403 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 171/353] 10.222.0.55 () {56 vars in 1180 bytes} [Fri Sep 23 14:58:49 2022] GET /identity/v3/groups/9980b9236dfe4cdabb8317a18b888217 => generated 226 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:58:49.574379 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6844fb4e-486c-484f-a3f6-042e46accc9e None None] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:49.574661 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6844fb4e-486c-484f-a3f6-042e46accc9e None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:49.574994 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6844fb4e-486c-484f-a3f6-042e46accc9e None None] PATH_INFO: `/` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:49.576205 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 172/354] 10.222.0.55 () {54 vars in 927 bytes} [Fri Sep 23 14:58:49 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 14:58:49.576907 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6844fb4e-486c-484f-a3f6-042e46accc9e None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:49.578620 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-882e7ffe-25d9-415c-bd16-384389c5d3bb None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 23 14:58:49.578620 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 14:58:49.578620 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 14:58:49.578620 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 14:58:49.578620 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 14:58:49.578620 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 14:58:49.578620 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 14:58:49.578620 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 14:58:49.578620 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 14:58:49.578620 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 14:58:49.578620 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 14:58:49.578620 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 14:58:49.578620 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 14:58:49.578620 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 14:58:49.578620 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 14:58:49.578620 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 14:58:49.579704 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 14:58:49.579704 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 14:58:49.579704 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 14:58:49.579704 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 14:58:49.579704 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 14:58:49.579704 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 14:58:49.579704 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 14:58:49.579704 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 14:58:49.579704 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 14:58:49.579704 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 14:58:49.579704 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 14:58:49.579704 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Sep 23 14:58:49.579704 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 14:58:49.579704 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Sep 23 14:58:49.579704 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 14:58:49.579704 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 14:58:49.580757 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 14:58:49.580757 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Sep 23 14:58:49.580757 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 14:58:49.580757 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Sep 23 14:58:49.580757 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 14:58:49.580757 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 14:58:49.580757 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 14:58:49.580757 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 14:58:49.580757 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 14:58:49.580757 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 14:58:49.580757 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 23 14:58:49.580757 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Sep 23 14:58:49.580757 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 183/355] 10.222.0.55 () {56 vars in 1099 bytes} [Fri Sep 23 14:58:49 2022] GET /identity/v3/roles/reader => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 14:58:49.581520 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882e7ffe-25d9-415c-bd16-384389c5d3bb None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:49.689834 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:49.703723 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882e7ffe-25d9-415c-bd16-384389c5d3bb None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:49.709131 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6844fb4e-486c-484f-a3f6-042e46accc9e None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:49.742830 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0b2228-f2f0-4839-b342-1977b4293122 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:49.744645 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0b2228-f2f0-4839-b342-1977b4293122 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:49.744932 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0b2228-f2f0-4839-b342-1977b4293122 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:49.745203 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0b2228-f2f0-4839-b342-1977b4293122 None admin] PATH_INFO: `/v3/projects/0ffbe3ca86954853be08ecf627d6af00` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:49.746630 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0b2228-f2f0-4839-b342-1977b4293122 None admin] RBAC: Authorizing `identity:get_project(project_id=0ffbe3ca86954853be08ecf627d6af00)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:49.747838 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0b2228-f2f0-4839-b342-1977b4293122 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:49.749514 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 184/356] 10.222.0.55 () {56 vars in 1186 bytes} [Fri Sep 23 14:58:49 2022] GET /identity/v3/projects/0ffbe3ca86954853be08ecf627d6af00 => generated 298 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:58:49.750659 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b435d978-105d-4243-9fdf-a5656bd6102b None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:49.750941 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b435d978-105d-4243-9fdf-a5656bd6102b None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:49.751216 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b435d978-105d-4243-9fdf-a5656bd6102b None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:49.752464 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 185/357] 10.222.0.55 () {54 vars in 927 bytes} [Fri Sep 23 14:58:49 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 14:58:49.753157 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b435d978-105d-4243-9fdf-a5656bd6102b None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:49.753852 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06ee267-bec7-4be4-8291-0d8b05631e04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:49.754654 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06ee267-bec7-4be4-8291-0d8b05631e04 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:49.755202 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06ee267-bec7-4be4-8291-0d8b05631e04 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:49.755630 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06ee267-bec7-4be4-8291-0d8b05631e04 None admin] PATH_INFO: `/v3/projects/0ffbe3ca86954853be08ecf627d6af00` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:49.756972 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06ee267-bec7-4be4-8291-0d8b05631e04 None admin] RBAC: Authorizing `identity:get_project(project_id=0ffbe3ca86954853be08ecf627d6af00)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:49.758731 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06ee267-bec7-4be4-8291-0d8b05631e04 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:49.764061 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 173/358] 10.222.0.55 () {56 vars in 1186 bytes} [Fri Sep 23 14:58:49 2022] GET /identity/v3/projects/0ffbe3ca86954853be08ecf627d6af00 => generated 298 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:58:49.764061 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9f0f3d-3237-4a75-917d-616d881ba44b None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:49.764061 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9f0f3d-3237-4a75-917d-616d881ba44b None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:49.764061 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9f0f3d-3237-4a75-917d-616d881ba44b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:49.782585 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-9a9f0f3d-3237-4a75-917d-616d881ba44b None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 14:58:49.783779 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-9a9f0f3d-3237-4a75-917d-616d881ba44b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 14:58:49.803567 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:49.865427 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 174/359] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Sep 23 14:58:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 14:58:49.866468 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9f0f3d-3237-4a75-917d-616d881ba44b None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:49.878753 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b435d978-105d-4243-9fdf-a5656bd6102b None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:49.924473 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be296660-6cd7-4530-91a6-9427530ddc5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:49.925103 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be296660-6cd7-4530-91a6-9427530ddc5a None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:49.925688 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be296660-6cd7-4530-91a6-9427530ddc5a None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:49.925980 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be296660-6cd7-4530-91a6-9427530ddc5a None admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:49.926982 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be296660-6cd7-4530-91a6-9427530ddc5a None admin] RBAC: Adding query filter params (name=reader) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:58:49.927286 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be296660-6cd7-4530-91a6-9427530ddc5a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:49.929696 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be296660-6cd7-4530-91a6-9427530ddc5a None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:49.941860 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 186/360] 10.222.0.55 () {56 vars in 1101 bytes} [Fri Sep 23 14:58:49 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:58:49.943150 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be296660-6cd7-4530-91a6-9427530ddc5a None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:49.978675 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:49.996459 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9f0f3d-3237-4a75-917d-616d881ba44b None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:50.041784 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a113aeb2-f530-4ef9-96b9-ef9e0f69b702 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:50.042845 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a113aeb2-f530-4ef9-96b9-ef9e0f69b702 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:50.043300 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a113aeb2-f530-4ef9-96b9-ef9e0f69b702 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:50.043744 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a113aeb2-f530-4ef9-96b9-ef9e0f69b702 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:50.044832 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a113aeb2-f530-4ef9-96b9-ef9e0f69b702 None admin] RBAC: Adding query filter params (group.id=9980b9236dfe4cdabb8317a18b888217, role.id=af257be2dee54d6b942f58650196582c, scope.project.id=346756b911774d55af2a9ad3de2143b4) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:58:50.045151 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a113aeb2-f530-4ef9-96b9-ef9e0f69b702 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:50.046786 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a113aeb2-f530-4ef9-96b9-ef9e0f69b702 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:50.053170 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 175/361] 10.222.0.55 () {56 vars in 1376 bytes} [Fri Sep 23 14:58:49 2022] GET /identity/v3/role_assignments?group.id=9980b9236dfe4cdabb8317a18b888217&scope.project.id=346756b911774d55af2a9ad3de2143b4&role.id=af257be2dee54d6b942f58650196582c => generated 260 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:58:50.054442 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a408f27-c504-4d81-9939-9a4f9618e077 None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:50.054779 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a408f27-c504-4d81-9939-9a4f9618e077 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:50.055060 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a408f27-c504-4d81-9939-9a4f9618e077 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:50.078883 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-5a408f27-c504-4d81-9939-9a4f9618e077 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 14:58:50.080152 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-5a408f27-c504-4d81-9939-9a4f9618e077 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 14:58:50.087553 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be296660-6cd7-4530-91a6-9427530ddc5a None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:50.089864 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:50.134888 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697a0394-1bc0-4d7a-8b30-663d23e4d254 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:50.137226 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697a0394-1bc0-4d7a-8b30-663d23e4d254 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:50.137964 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697a0394-1bc0-4d7a-8b30-663d23e4d254 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:50.138973 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697a0394-1bc0-4d7a-8b30-663d23e4d254 None admin] PATH_INFO: `/v3/groups/9980b9236dfe4cdabb8317a18b888217` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:50.140910 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697a0394-1bc0-4d7a-8b30-663d23e4d254 None admin] RBAC: Authorizing `identity:get_group(group_id=9980b9236dfe4cdabb8317a18b888217)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:50.142601 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697a0394-1bc0-4d7a-8b30-663d23e4d254 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:50.144993 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 187/362] 10.222.0.55 () {56 vars in 1180 bytes} [Fri Sep 23 14:58:49 2022] GET /identity/v3/groups/9980b9236dfe4cdabb8317a18b888217 => generated 226 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:58:50.145821 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697a0394-1bc0-4d7a-8b30-663d23e4d254 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:50.172579 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 176/363] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Sep 23 14:58:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 14:58:50.174137 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a408f27-c504-4d81-9939-9a4f9618e077 None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:50.283594 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697a0394-1bc0-4d7a-8b30-663d23e4d254 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:50.297418 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:50.327524 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a408f27-c504-4d81-9939-9a4f9618e077 None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:50.333387 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4707b14f-98de-4946-a5d7-6daaa3f25226 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:50.334034 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4707b14f-98de-4946-a5d7-6daaa3f25226 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:50.334316 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4707b14f-98de-4946-a5d7-6daaa3f25226 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:50.334590 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4707b14f-98de-4946-a5d7-6daaa3f25226 None admin] PATH_INFO: `/v3/groups/9980b9236dfe4cdabb8317a18b888217` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:50.335695 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4707b14f-98de-4946-a5d7-6daaa3f25226 None admin] RBAC: Authorizing `identity:get_group(group_id=9980b9236dfe4cdabb8317a18b888217)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:50.339487 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4707b14f-98de-4946-a5d7-6daaa3f25226 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:50.342311 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 188/364] 10.222.0.55 () {56 vars in 1180 bytes} [Fri Sep 23 14:58:50 2022] GET /identity/v3/groups/9980b9236dfe4cdabb8317a18b888217 => generated 226 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:58:50.344482 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4707b14f-98de-4946-a5d7-6daaa3f25226 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:50.379953 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb9f4d7-e9dc-4705-a22a-a4b8ed8a8fb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:50.381273 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb9f4d7-e9dc-4705-a22a-a4b8ed8a8fb9 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:50.382204 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb9f4d7-e9dc-4705-a22a-a4b8ed8a8fb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:50.382953 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb9f4d7-e9dc-4705-a22a-a4b8ed8a8fb9 None admin] PATH_INFO: `/v3/users/71026b5a24794d399d38bbd4bf2eb92b` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:50.385740 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb9f4d7-e9dc-4705-a22a-a4b8ed8a8fb9 None admin] RBAC: Authorizing `identity:get_user(user_id=71026b5a24794d399d38bbd4bf2eb92b)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:50.388074 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb9f4d7-e9dc-4705-a22a-a4b8ed8a8fb9 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:50.390647 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 177/365] 10.222.0.55 () {56 vars in 1177 bytes} [Fri Sep 23 14:58:50 2022] GET /identity/v3/users/71026b5a24794d399d38bbd4bf2eb92b => generated 293 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:58:50.391942 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb9f4d7-e9dc-4705-a22a-a4b8ed8a8fb9 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:50.399603 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:50.487948 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4707b14f-98de-4946-a5d7-6daaa3f25226 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:50.535167 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb9f4d7-e9dc-4705-a22a-a4b8ed8a8fb9 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:50.537077 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ae6f6a-128f-400f-92ad-bf90964b41a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:50.538817 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ae6f6a-128f-400f-92ad-bf90964b41a0 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:50.539334 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ae6f6a-128f-400f-92ad-bf90964b41a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:50.539811 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ae6f6a-128f-400f-92ad-bf90964b41a0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:50.541094 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ae6f6a-128f-400f-92ad-bf90964b41a0 None admin] RBAC: Adding query filter params (group.id=9980b9236dfe4cdabb8317a18b888217, role.id=5c74fdc0e9cc44538cd8282006ed4443, scope.project.id=0ffbe3ca86954853be08ecf627d6af00) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:58:50.541632 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ae6f6a-128f-400f-92ad-bf90964b41a0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:50.544125 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ae6f6a-128f-400f-92ad-bf90964b41a0 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:50.551809 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 189/366] 10.222.0.55 () {56 vars in 1376 bytes} [Fri Sep 23 14:58:50 2022] GET /identity/v3/role_assignments?group.id=9980b9236dfe4cdabb8317a18b888217&scope.project.id=0ffbe3ca86954853be08ecf627d6af00&role.id=5c74fdc0e9cc44538cd8282006ed4443 => generated 260 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:58:50.552953 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ae6f6a-128f-400f-92ad-bf90964b41a0 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:50.579923 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfae145f-a8d5-4c6b-bcce-64f8f8cea021 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:50.580800 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfae145f-a8d5-4c6b-bcce-64f8f8cea021 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:50.581184 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfae145f-a8d5-4c6b-bcce-64f8f8cea021 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:50.581559 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfae145f-a8d5-4c6b-bcce-64f8f8cea021 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:50.595571 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfae145f-a8d5-4c6b-bcce-64f8f8cea021 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 14:58:50.601595 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application [None req-dfae145f-a8d5-4c6b-bcce-64f8f8cea021 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 23 14:58:50.601595 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 14:58:50.601595 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 14:58:50.601595 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 14:58:50.601595 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 14:58:50.601595 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 14:58:50.601595 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 14:58:50.601595 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 14:58:50.601595 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 14:58:50.601595 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 14:58:50.601595 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 14:58:50.601595 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 14:58:50.601595 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 14:58:50.601595 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 14:58:50.601595 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 14:58:50.601595 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 14:58:50.602599 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 14:58:50.602599 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 14:58:50.602599 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 14:58:50.602599 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 14:58:50.602599 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 14:58:50.602599 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 14:58:50.602599 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 14:58:50.602599 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 14:58:50.602599 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 14:58:50.602599 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 14:58:50.602599 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 14:58:50.602599 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application with Lock( Sep 23 14:58:50.602599 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 14:58:50.602599 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._enter() Sep 23 14:58:50.602599 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 14:58:50.602599 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 14:58:50.603460 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 14:58:50.603460 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.creator() Sep 23 14:58:50.603460 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 14:58:50.603460 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application created_value = creator( Sep 23 14:58:50.603460 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 14:58:50.603460 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 14:58:50.603460 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 14:58:50.603460 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 14:58:50.603460 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 14:58:50.603460 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 14:58:50.603460 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 23 14:58:50.603460 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Sep 23 14:58:50.603460 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 178/367] 10.222.0.55 () {56 vars in 1114 bytes} [Fri Sep 23 14:58:50 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 14:58:50.605074 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfae145f-a8d5-4c6b-bcce-64f8f8cea021 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:50.615796 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:50.709281 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ae6f6a-128f-400f-92ad-bf90964b41a0 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:50.714867 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:50.765276 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfae145f-a8d5-4c6b-bcce-64f8f8cea021 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:50.771042 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa873d9c-db69-4023-b616-df9a64a5f1a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:50.771678 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa873d9c-db69-4023-b616-df9a64a5f1a8 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:50.771964 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa873d9c-db69-4023-b616-df9a64a5f1a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:50.776484 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa873d9c-db69-4023-b616-df9a64a5f1a8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:50.779237 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa873d9c-db69-4023-b616-df9a64a5f1a8 None admin] RBAC: Adding query filter params (group.id=9980b9236dfe4cdabb8317a18b888217, role.id=af257be2dee54d6b942f58650196582c, scope.project.id=0ffbe3ca86954853be08ecf627d6af00) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:58:50.779818 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa873d9c-db69-4023-b616-df9a64a5f1a8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:50.781534 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa873d9c-db69-4023-b616-df9a64a5f1a8 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:50.790051 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 190/368] 10.222.0.55 () {56 vars in 1376 bytes} [Fri Sep 23 14:58:50 2022] GET /identity/v3/role_assignments?group.id=9980b9236dfe4cdabb8317a18b888217&scope.project.id=0ffbe3ca86954853be08ecf627d6af00&role.id=af257be2dee54d6b942f58650196582c => generated 260 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:58:50.791363 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa873d9c-db69-4023-b616-df9a64a5f1a8 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:50.820902 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2defeb78-f31f-4e25-a6df-9fcb82c51fc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:50.822209 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2defeb78-f31f-4e25-a6df-9fcb82c51fc2 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:50.822516 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2defeb78-f31f-4e25-a6df-9fcb82c51fc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:50.822791 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2defeb78-f31f-4e25-a6df-9fcb82c51fc2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:50.823569 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2defeb78-f31f-4e25-a6df-9fcb82c51fc2 None admin] RBAC: Adding query filter params (role.id=ea002840090d4194bb75fa083a55dc5f, user.id=71026b5a24794d399d38bbd4bf2eb92b, scope.system=all) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:58:50.823878 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2defeb78-f31f-4e25-a6df-9fcb82c51fc2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:50.825605 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2defeb78-f31f-4e25-a6df-9fcb82c51fc2 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:50.842622 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 179/369] 10.222.0.55 () {56 vars in 1308 bytes} [Fri Sep 23 14:58:50 2022] GET /identity/v3/role_assignments?user.id=71026b5a24794d399d38bbd4bf2eb92b&scope.system=all&role.id=ea002840090d4194bb75fa083a55dc5f => generated 510 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:58:50.843512 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2defeb78-f31f-4e25-a6df-9fcb82c51fc2 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:50.940317 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa873d9c-db69-4023-b616-df9a64a5f1a8 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:50.988517 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:50.992726 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd30ebc3-1464-4986-bb13-5f9d8419b49e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:50.994153 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd30ebc3-1464-4986-bb13-5f9d8419b49e None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:50.994535 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd30ebc3-1464-4986-bb13-5f9d8419b49e None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:50.994900 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd30ebc3-1464-4986-bb13-5f9d8419b49e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:51.026397 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd30ebc3-1464-4986-bb13-5f9d8419b49e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 14:58:51.036893 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-cd30ebc3-1464-4986-bb13-5f9d8419b49e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 23 14:58:51.036893 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 14:58:51.036893 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 14:58:51.036893 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 14:58:51.036893 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 14:58:51.036893 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 14:58:51.036893 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 14:58:51.036893 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 14:58:51.036893 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 14:58:51.036893 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 14:58:51.036893 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 14:58:51.036893 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 14:58:51.036893 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 14:58:51.036893 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 14:58:51.036893 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 14:58:51.036893 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 14:58:51.037888 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 14:58:51.037888 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 14:58:51.037888 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 14:58:51.037888 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 14:58:51.037888 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 14:58:51.037888 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 14:58:51.037888 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 14:58:51.037888 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 14:58:51.037888 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 14:58:51.037888 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 14:58:51.037888 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 14:58:51.037888 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Sep 23 14:58:51.037888 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 14:58:51.037888 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Sep 23 14:58:51.037888 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 14:58:51.037888 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 14:58:51.038783 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 14:58:51.038783 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Sep 23 14:58:51.038783 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 14:58:51.038783 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Sep 23 14:58:51.038783 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 14:58:51.038783 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 14:58:51.038783 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 14:58:51.038783 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 14:58:51.038783 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 14:58:51.038783 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 14:58:51.038783 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 23 14:58:51.038783 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Sep 23 14:58:51.038783 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 191/370] 10.222.0.55 () {56 vars in 1096 bytes} [Fri Sep 23 14:58:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 14:58:51.053710 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd30ebc3-1464-4986-bb13-5f9d8419b49e None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:51.086021 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2defeb78-f31f-4e25-a6df-9fcb82c51fc2 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:51.105100 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:51.163088 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8772b6b7-e487-4318-8878-dd6064a8ca98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:51.163707 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8772b6b7-e487-4318-8878-dd6064a8ca98 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:51.164009 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8772b6b7-e487-4318-8878-dd6064a8ca98 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:51.164274 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8772b6b7-e487-4318-8878-dd6064a8ca98 None admin] PATH_INFO: `/v3/roles` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:51.165497 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8772b6b7-e487-4318-8878-dd6064a8ca98 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:58:51.167835 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8772b6b7-e487-4318-8878-dd6064a8ca98 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:51.167835 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8772b6b7-e487-4318-8878-dd6064a8ca98 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:51.190140 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 180/371] 10.222.0.55 () {56 vars in 1111 bytes} [Fri Sep 23 14:58:50 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 347 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:58:51.195784 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8772b6b7-e487-4318-8878-dd6064a8ca98 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:51.217361 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd30ebc3-1464-4986-bb13-5f9d8419b49e None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:51.263760 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a0bd68-33ef-4aa3-ac37-b9ed50b6bd04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:51.264532 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a0bd68-33ef-4aa3-ac37-b9ed50b6bd04 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:51.264731 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a0bd68-33ef-4aa3-ac37-b9ed50b6bd04 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:51.264938 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a0bd68-33ef-4aa3-ac37-b9ed50b6bd04 None admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:51.265606 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a0bd68-33ef-4aa3-ac37-b9ed50b6bd04 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:58:51.265908 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a0bd68-33ef-4aa3-ac37-b9ed50b6bd04 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:51.267292 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a0bd68-33ef-4aa3-ac37-b9ed50b6bd04 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:51.277994 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 192/372] 10.222.0.55 () {56 vars in 1099 bytes} [Fri Sep 23 14:58:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:58:51.283722 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a0bd68-33ef-4aa3-ac37-b9ed50b6bd04 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:51.396642 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8772b6b7-e487-4318-8878-dd6064a8ca98 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:51.446232 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:51.451235 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:51.464838 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a0bd68-33ef-4aa3-ac37-b9ed50b6bd04 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:51.468889 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42051251-2198-4a04-831c-696fe1214b74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:51.469594 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42051251-2198-4a04-831c-696fe1214b74 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:51.469948 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42051251-2198-4a04-831c-696fe1214b74 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:51.470294 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42051251-2198-4a04-831c-696fe1214b74 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:51.508306 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051673d8-ae56-40a4-a7e9-28be0c7c272e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:51.509540 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051673d8-ae56-40a4-a7e9-28be0c7c272e None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:51.510097 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051673d8-ae56-40a4-a7e9-28be0c7c272e None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:51.510552 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051673d8-ae56-40a4-a7e9-28be0c7c272e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:51.587865 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42051251-2198-4a04-831c-696fe1214b74 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:51.589770 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42051251-2198-4a04-831c-696fe1214b74 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:51.611543 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051673d8-ae56-40a4-a7e9-28be0c7c272e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:51.613155 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051673d8-ae56-40a4-a7e9-28be0c7c272e None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:51.717189 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 181/373] 10.222.0.55 () {58 vars in 1282 bytes} [Fri Sep 23 14:58:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 522 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 23 14:58:51.721476 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:51.724428 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42051251-2198-4a04-831c-696fe1214b74 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:51.736257 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 193/374] 10.222.0.55 () {58 vars in 1282 bytes} [Fri Sep 23 14:58:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 453 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 23 14:58:51.741893 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051673d8-ae56-40a4-a7e9-28be0c7c272e None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:51.754145 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:51.834126 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42051251-2198-4a04-831c-696fe1214b74 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:51.872554 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48baa2ef-2688-4e9b-9eb7-bdcda7761e05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:51.873565 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48baa2ef-2688-4e9b-9eb7-bdcda7761e05 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:51.874034 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48baa2ef-2688-4e9b-9eb7-bdcda7761e05 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:51.874505 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48baa2ef-2688-4e9b-9eb7-bdcda7761e05 None admin] PATH_INFO: `/v3/projects/346756b911774d55af2a9ad3de2143b4` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:51.875833 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48baa2ef-2688-4e9b-9eb7-bdcda7761e05 None admin] RBAC: Authorizing `identity:get_project(project_id=346756b911774d55af2a9ad3de2143b4)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:51.877491 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48baa2ef-2688-4e9b-9eb7-bdcda7761e05 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:51.879322 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 182/375] 10.222.0.55 () {56 vars in 1186 bytes} [Fri Sep 23 14:58:51 2022] GET /identity/v3/projects/346756b911774d55af2a9ad3de2143b4 => generated 302 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 14:58:51.885562 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48baa2ef-2688-4e9b-9eb7-bdcda7761e05 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:51.914276 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051673d8-ae56-40a4-a7e9-28be0c7c272e None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:51.981651 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67af49f1-bcb6-4730-92c6-44f5e298f8cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:51.982282 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67af49f1-bcb6-4730-92c6-44f5e298f8cb None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:51.982568 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67af49f1-bcb6-4730-92c6-44f5e298f8cb None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:51.982845 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67af49f1-bcb6-4730-92c6-44f5e298f8cb None admin] PATH_INFO: `/v3/projects/23b5c58d198d4456bfeec0a8daae1113` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:51.984662 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67af49f1-bcb6-4730-92c6-44f5e298f8cb None admin] RBAC: Authorizing `identity:get_project(project_id=23b5c58d198d4456bfeec0a8daae1113)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:51.986099 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67af49f1-bcb6-4730-92c6-44f5e298f8cb None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:51.987930 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 194/376] 10.222.0.55 () {56 vars in 1186 bytes} [Fri Sep 23 14:58:51 2022] GET /identity/v3/projects/23b5c58d198d4456bfeec0a8daae1113 => generated 344 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 14:58:51.992330 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:51.996989 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67af49f1-bcb6-4730-92c6-44f5e298f8cb None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:51.999401 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48baa2ef-2688-4e9b-9eb7-bdcda7761e05 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:52.041292 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f585185-8acd-449c-b8b6-520f1c625986 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:52.042107 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f585185-8acd-449c-b8b6-520f1c625986 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:52.042482 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f585185-8acd-449c-b8b6-520f1c625986 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:52.042853 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f585185-8acd-449c-b8b6-520f1c625986 None admin] PATH_INFO: `/v3/groups/9980b9236dfe4cdabb8317a18b888217` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:52.044247 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f585185-8acd-449c-b8b6-520f1c625986 None admin] RBAC: Authorizing `identity:get_group(group_id=9980b9236dfe4cdabb8317a18b888217)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:52.045881 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f585185-8acd-449c-b8b6-520f1c625986 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:52.047934 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 183/377] 10.222.0.55 () {56 vars in 1180 bytes} [Fri Sep 23 14:58:51 2022] GET /identity/v3/groups/9980b9236dfe4cdabb8317a18b888217 => generated 226 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 14:58:52.053791 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f585185-8acd-449c-b8b6-520f1c625986 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:52.098064 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:52.164821 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f585185-8acd-449c-b8b6-520f1c625986 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:52.171375 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67af49f1-bcb6-4730-92c6-44f5e298f8cb None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:52.217393 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c33554-d45c-4d45-939e-e126e24e7585 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:52.218116 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c33554-d45c-4d45-939e-e126e24e7585 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:52.218532 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c33554-d45c-4d45-939e-e126e24e7585 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:52.218915 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c33554-d45c-4d45-939e-e126e24e7585 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:52.219707 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c33554-d45c-4d45-939e-e126e24e7585 None admin] RBAC: Adding query filter params (group.id=9980b9236dfe4cdabb8317a18b888217, role.id=5c74fdc0e9cc44538cd8282006ed4443, scope.project.id=346756b911774d55af2a9ad3de2143b4) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:58:52.220114 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c33554-d45c-4d45-939e-e126e24e7585 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:52.221672 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c33554-d45c-4d45-939e-e126e24e7585 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:52.222088 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15f2191-27c7-4ba5-8199-e2050c07f613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:52.222743 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15f2191-27c7-4ba5-8199-e2050c07f613 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:52.223029 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15f2191-27c7-4ba5-8199-e2050c07f613 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:52.223308 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15f2191-27c7-4ba5-8199-e2050c07f613 None admin] PATH_INFO: `/v3/groups/0f32957856bc433188452103c17b1175` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:52.230640 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 184/378] 10.222.0.55 () {56 vars in 1376 bytes} [Fri Sep 23 14:58:52 2022] GET /identity/v3/role_assignments?group.id=9980b9236dfe4cdabb8317a18b888217&scope.project.id=346756b911774d55af2a9ad3de2143b4&role.id=5c74fdc0e9cc44538cd8282006ed4443 => generated 260 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 14:58:52.241029 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15f2191-27c7-4ba5-8199-e2050c07f613 None admin] RBAC: Authorizing `identity:get_group(group_id=0f32957856bc433188452103c17b1175)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:52.242469 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15f2191-27c7-4ba5-8199-e2050c07f613 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:52.245465 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 195/379] 10.222.0.55 () {56 vars in 1180 bytes} [Fri Sep 23 14:58:51 2022] GET /identity/v3/groups/0f32957856bc433188452103c17b1175 => generated 229 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:58:52.252285 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c33554-d45c-4d45-939e-e126e24e7585 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:52.299094 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:52.390377 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c33554-d45c-4d45-939e-e126e24e7585 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:52.431601 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df1ddcd-0b19-49e0-9e00-bd2423d2174c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:52.432433 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df1ddcd-0b19-49e0-9e00-bd2423d2174c None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:52.432717 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df1ddcd-0b19-49e0-9e00-bd2423d2174c None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:52.432984 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df1ddcd-0b19-49e0-9e00-bd2423d2174c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:52.433785 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df1ddcd-0b19-49e0-9e00-bd2423d2174c None admin] RBAC: Adding query filter params (group.id=0f32957856bc433188452103c17b1175, role.id=8e2919e79df040e8926f21607818b58c, scope.project.id=23b5c58d198d4456bfeec0a8daae1113) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:58:52.434094 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df1ddcd-0b19-49e0-9e00-bd2423d2174c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:52.435625 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df1ddcd-0b19-49e0-9e00-bd2423d2174c None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:52.443916 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 185/380] 10.222.0.55 () {56 vars in 1376 bytes} [Fri Sep 23 14:58:52 2022] GET /identity/v3/role_assignments?group.id=0f32957856bc433188452103c17b1175&scope.project.id=23b5c58d198d4456bfeec0a8daae1113&role.id=8e2919e79df040e8926f21607818b58c => generated 260 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:58:52.787169 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dada58b-427a-44a1-8180-02756b3088bb None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:52.787582 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dada58b-427a-44a1-8180-02756b3088bb None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:52.787871 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dada58b-427a-44a1-8180-02756b3088bb None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:52.789482 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 196/381] 10.222.0.55 () {54 vars in 927 bytes} [Fri Sep 23 14:58:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 14:58:52.805934 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb59bfe-3479-4bdb-bc5b-a57d111cd920 None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:52.806393 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb59bfe-3479-4bdb-bc5b-a57d111cd920 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:52.806883 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb59bfe-3479-4bdb-bc5b-a57d111cd920 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:52.837085 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-0eb59bfe-3479-4bdb-bc5b-a57d111cd920 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 14:58:52.838388 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-0eb59bfe-3479-4bdb-bc5b-a57d111cd920 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 14:58:52.930619 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 186/382] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Sep 23 14:58:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 14:58:53.268518 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dada58b-427a-44a1-8180-02756b3088bb None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:53.409662 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dada58b-427a-44a1-8180-02756b3088bb None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:53.433452 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:53.456959 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac254ab-8e97-4917-b243-8d3834c75dd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:53.457743 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac254ab-8e97-4917-b243-8d3834c75dd8 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:53.458063 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac254ab-8e97-4917-b243-8d3834c75dd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:53.458387 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac254ab-8e97-4917-b243-8d3834c75dd8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:53.481464 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac254ab-8e97-4917-b243-8d3834c75dd8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 14:58:53.490019 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-eac254ab-8e97-4917-b243-8d3834c75dd8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 23 14:58:53.490019 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 14:58:53.490019 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 14:58:53.490019 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 14:58:53.490019 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 14:58:53.490019 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 14:58:53.490019 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 14:58:53.490019 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 14:58:53.490019 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 14:58:53.490019 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 14:58:53.490019 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 14:58:53.490019 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 14:58:53.490019 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 14:58:53.490019 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 14:58:53.490019 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 14:58:53.490019 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 14:58:53.490979 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 14:58:53.490979 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 14:58:53.490979 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 14:58:53.490979 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 14:58:53.490979 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 14:58:53.490979 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 14:58:53.490979 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 14:58:53.490979 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 14:58:53.490979 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 14:58:53.490979 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 14:58:53.490979 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 14:58:53.490979 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Sep 23 14:58:53.490979 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 14:58:53.490979 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Sep 23 14:58:53.490979 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 14:58:53.490979 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 14:58:53.492255 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 14:58:53.492255 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Sep 23 14:58:53.492255 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 14:58:53.492255 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Sep 23 14:58:53.492255 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 14:58:53.492255 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 14:58:53.492255 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 14:58:53.492255 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 14:58:53.492255 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 14:58:53.492255 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 14:58:53.492255 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 23 14:58:53.492255 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Sep 23 14:58:53.492255 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 197/383] 10.222.0.55 () {56 vars in 1099 bytes} [Fri Sep 23 14:58:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 14:58:53.499277 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb59bfe-3479-4bdb-bc5b-a57d111cd920 None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:53.536792 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:53.639283 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f438bb5-1aa9-4dd0-91e1-40b78d5f1166 None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:53.640061 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f438bb5-1aa9-4dd0-91e1-40b78d5f1166 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:53.640735 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f438bb5-1aa9-4dd0-91e1-40b78d5f1166 None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:53.644522 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 198/384] 10.222.0.55 () {54 vars in 927 bytes} [Fri Sep 23 14:58:53 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 23 14:58:53.650869 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b61df34-827b-48f5-b58e-e12a0e5265c1 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:53.651570 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b61df34-827b-48f5-b58e-e12a0e5265c1 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:53.652170 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b61df34-827b-48f5-b58e-e12a0e5265c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:53.663756 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb59bfe-3479-4bdb-bc5b-a57d111cd920 None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:53.678490 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-8b61df34-827b-48f5-b58e-e12a0e5265c1 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 14:58:53.680259 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-8b61df34-827b-48f5-b58e-e12a0e5265c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 14:58:53.725815 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cdf5a5-02ac-4699-8812-afe3b0f7f4db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:53.726435 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cdf5a5-02ac-4699-8812-afe3b0f7f4db None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:53.726749 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cdf5a5-02ac-4699-8812-afe3b0f7f4db None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:53.727019 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cdf5a5-02ac-4699-8812-afe3b0f7f4db None admin] PATH_INFO: `/v3/roles` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:53.728328 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cdf5a5-02ac-4699-8812-afe3b0f7f4db None admin] RBAC: Adding query filter params (name=member) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:58:53.728627 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cdf5a5-02ac-4699-8812-afe3b0f7f4db None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:53.730018 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cdf5a5-02ac-4699-8812-afe3b0f7f4db None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:53.754246 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 187/385] 10.222.0.55 () {56 vars in 1101 bytes} [Fri Sep 23 14:58:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:58:53.755329 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cdf5a5-02ac-4699-8812-afe3b0f7f4db None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:53.814334 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 199/386] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Sep 23 14:58:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 23 14:58:53.874688 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:53.901100 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cdf5a5-02ac-4699-8812-afe3b0f7f4db None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:53.945228 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f696b45-6935-4459-a0aa-13ed0e2712a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:53.945804 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f696b45-6935-4459-a0aa-13ed0e2712a7 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:53.946078 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f696b45-6935-4459-a0aa-13ed0e2712a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:53.946436 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f696b45-6935-4459-a0aa-13ed0e2712a7 None admin] PATH_INFO: `/v3/groups/9980b9236dfe4cdabb8317a18b888217` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:53.947655 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f696b45-6935-4459-a0aa-13ed0e2712a7 None admin] RBAC: Authorizing `identity:get_group(group_id=9980b9236dfe4cdabb8317a18b888217)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:53.949221 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f696b45-6935-4459-a0aa-13ed0e2712a7 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:53.951178 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 188/387] 10.222.0.55 () {56 vars in 1180 bytes} [Fri Sep 23 14:58:53 2022] GET /identity/v3/groups/9980b9236dfe4cdabb8317a18b888217 => generated 226 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:58:53.956900 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b61df34-827b-48f5-b58e-e12a0e5265c1 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:53.985255 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd27a93d-99e6-4c8f-869c-9d5df032add0 None None] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:53.985559 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd27a93d-99e6-4c8f-869c-9d5df032add0 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:53.985801 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd27a93d-99e6-4c8f-869c-9d5df032add0 None None] PATH_INFO: `/` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:53.987169 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 189/388] 10.222.0.55 () {54 vars in 927 bytes} [Fri Sep 23 14:58:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 14:58:53.992202 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc3c6f2-da60-4958-bbbe-b1b16d682533 None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:53.992529 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc3c6f2-da60-4958-bbbe-b1b16d682533 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:53.992804 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc3c6f2-da60-4958-bbbe-b1b16d682533 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:54.025733 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-4bc3c6f2-da60-4958-bbbe-b1b16d682533 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 14:58:54.028881 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-4bc3c6f2-da60-4958-bbbe-b1b16d682533 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 14:58:54.113592 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b61df34-827b-48f5-b58e-e12a0e5265c1 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:54.131091 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:54.153284 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bdf0f7-7844-4fca-b77f-dfcae61b6dbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:54.154483 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bdf0f7-7844-4fca-b77f-dfcae61b6dbe None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:54.154966 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bdf0f7-7844-4fca-b77f-dfcae61b6dbe None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:54.155439 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bdf0f7-7844-4fca-b77f-dfcae61b6dbe None admin] PATH_INFO: `/v3/projects/346756b911774d55af2a9ad3de2143b4` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:54.156908 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bdf0f7-7844-4fca-b77f-dfcae61b6dbe None admin] RBAC: Authorizing `identity:get_project(project_id=346756b911774d55af2a9ad3de2143b4)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:54.158807 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bdf0f7-7844-4fca-b77f-dfcae61b6dbe None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:54.161284 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 200/389] 10.222.0.55 () {56 vars in 1186 bytes} [Fri Sep 23 14:58:53 2022] GET /identity/v3/projects/346756b911774d55af2a9ad3de2143b4 => generated 302 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:58:54.162341 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 190/390] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Sep 23 14:58:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 23 14:58:54.170605 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bdf0f7-7844-4fca-b77f-dfcae61b6dbe None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:54.253909 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc3c6f2-da60-4958-bbbe-b1b16d682533 None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:54.329190 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bdf0f7-7844-4fca-b77f-dfcae61b6dbe None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:54.394052 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:54.415552 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc3c6f2-da60-4958-bbbe-b1b16d682533 None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:54.421661 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45088ebd-a719-427b-9079-1fefed9b3489 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:54.422358 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45088ebd-a719-427b-9079-1fefed9b3489 None admin] REQUEST_METHOD: `PUT` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:54.422641 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45088ebd-a719-427b-9079-1fefed9b3489 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:54.422915 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45088ebd-a719-427b-9079-1fefed9b3489 None admin] PATH_INFO: `/v3/projects/346756b911774d55af2a9ad3de2143b4/groups/9980b9236dfe4cdabb8317a18b888217/roles/af257be2dee54d6b942f58650196582c` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:54.424897 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45088ebd-a719-427b-9079-1fefed9b3489 None admin] RBAC: Authorizing `identity:create_grant(project_id=346756b911774d55af2a9ad3de2143b4, group_id=9980b9236dfe4cdabb8317a18b888217, role_id=af257be2dee54d6b942f58650196582c)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:54.426353 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45088ebd-a719-427b-9079-1fefed9b3489 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:54.453234 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df841d32-d920-4397-8697-e200f7b2d32a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:54.454200 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df841d32-d920-4397-8697-e200f7b2d32a None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:54.454699 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df841d32-d920-4397-8697-e200f7b2d32a None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:54.455178 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df841d32-d920-4397-8697-e200f7b2d32a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:54.467100 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df841d32-d920-4397-8697-e200f7b2d32a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 14:58:54.472146 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application [None req-df841d32-d920-4397-8697-e200f7b2d32a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 23 14:58:54.472146 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 14:58:54.472146 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 14:58:54.472146 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 14:58:54.472146 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 14:58:54.472146 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 14:58:54.472146 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 14:58:54.472146 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 14:58:54.472146 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 14:58:54.472146 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 14:58:54.472146 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 14:58:54.472146 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 14:58:54.472146 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 14:58:54.472146 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 14:58:54.472146 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 14:58:54.472146 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 14:58:54.473083 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 14:58:54.473083 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 14:58:54.473083 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 14:58:54.473083 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 14:58:54.473083 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 14:58:54.473083 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 14:58:54.473083 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 14:58:54.473083 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 14:58:54.473083 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 14:58:54.473083 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 14:58:54.473083 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 14:58:54.473083 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application with Lock( Sep 23 14:58:54.473083 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 14:58:54.473083 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._enter() Sep 23 14:58:54.473083 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 14:58:54.473083 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 14:58:54.473872 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 14:58:54.473872 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.creator() Sep 23 14:58:54.473872 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 14:58:54.473872 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application created_value = creator( Sep 23 14:58:54.473872 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 14:58:54.473872 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 14:58:54.473872 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 14:58:54.473872 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 14:58:54.473872 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 14:58:54.473872 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 14:58:54.473872 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 23 14:58:54.473872 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Sep 23 14:58:54.479223 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 191/391] 10.222.0.55 () {56 vars in 1114 bytes} [Fri Sep 23 14:58:54 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 23 14:58:54.479223 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df841d32-d920-4397-8697-e200f7b2d32a None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:54.550925 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 201/392] 10.222.0.55 () {58 vars in 1442 bytes} [Fri Sep 23 14:58:54 2022] PUT /identity/v3/projects/346756b911774d55af2a9ad3de2143b4/groups/9980b9236dfe4cdabb8317a18b888217/roles/af257be2dee54d6b942f58650196582c => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 14:58:54.552394 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45088ebd-a719-427b-9079-1fefed9b3489 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:54.609355 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:54.654335 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df841d32-d920-4397-8697-e200f7b2d32a None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:54.695113 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45088ebd-a719-427b-9079-1fefed9b3489 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:54.700480 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7478882d-080b-4d79-84b1-a6ca0bd9eb65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:54.700480 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7478882d-080b-4d79-84b1-a6ca0bd9eb65 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:54.700775 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7478882d-080b-4d79-84b1-a6ca0bd9eb65 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:54.701018 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7478882d-080b-4d79-84b1-a6ca0bd9eb65 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:54.710503 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:54.714754 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7478882d-080b-4d79-84b1-a6ca0bd9eb65 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 14:58:54.720356 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application [None req-7478882d-080b-4d79-84b1-a6ca0bd9eb65 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 23 14:58:54.720356 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 14:58:54.720356 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 14:58:54.720356 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 14:58:54.720356 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 14:58:54.720356 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 14:58:54.720356 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 14:58:54.720356 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 14:58:54.720356 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 14:58:54.720356 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 14:58:54.720356 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 14:58:54.720356 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 14:58:54.720356 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 14:58:54.720356 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 14:58:54.720356 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 14:58:54.720356 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 14:58:54.721328 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 14:58:54.721328 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 14:58:54.721328 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 14:58:54.721328 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 14:58:54.721328 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 14:58:54.721328 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 14:58:54.721328 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 14:58:54.721328 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 14:58:54.721328 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 14:58:54.721328 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 14:58:54.721328 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 14:58:54.721328 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application with Lock( Sep 23 14:58:54.721328 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 14:58:54.721328 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._enter() Sep 23 14:58:54.721328 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 14:58:54.721328 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 14:58:54.722627 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 14:58:54.722627 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.creator() Sep 23 14:58:54.722627 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 14:58:54.722627 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application created_value = creator( Sep 23 14:58:54.722627 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 14:58:54.722627 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 14:58:54.722627 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 14:58:54.722627 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 14:58:54.722627 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 14:58:54.722627 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 14:58:54.722627 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 23 14:58:54.722627 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Sep 23 14:58:54.722627 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 192/393] 10.222.0.55 () {56 vars in 1099 bytes} [Fri Sep 23 14:58:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 14:58:54.728113 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7478882d-080b-4d79-84b1-a6ca0bd9eb65 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:54.754066 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d970b1-bc36-4c26-9ef1-0dc588c73d4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:54.755152 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d970b1-bc36-4c26-9ef1-0dc588c73d4a None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:54.755647 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d970b1-bc36-4c26-9ef1-0dc588c73d4a None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:54.756008 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d970b1-bc36-4c26-9ef1-0dc588c73d4a None admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:54.756816 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d970b1-bc36-4c26-9ef1-0dc588c73d4a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:58:54.760447 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d970b1-bc36-4c26-9ef1-0dc588c73d4a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:54.761888 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d970b1-bc36-4c26-9ef1-0dc588c73d4a None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:54.772344 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 202/394] 10.222.0.55 () {56 vars in 1111 bytes} [Fri Sep 23 14:58:54 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:58:54.779505 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d970b1-bc36-4c26-9ef1-0dc588c73d4a None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:54.866053 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7478882d-080b-4d79-84b1-a6ca0bd9eb65 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:54.897343 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:54.901608 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d970b1-bc36-4c26-9ef1-0dc588c73d4a None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:54.939879 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d17e6e-0c0f-45e0-99ff-7706c9da8eb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:54.940961 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d17e6e-0c0f-45e0-99ff-7706c9da8eb0 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:54.941480 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d17e6e-0c0f-45e0-99ff-7706c9da8eb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:54.941963 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d17e6e-0c0f-45e0-99ff-7706c9da8eb0 None admin] PATH_INFO: `/v3/groups/9980b9236dfe4cdabb8317a18b888217` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:54.943302 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d17e6e-0c0f-45e0-99ff-7706c9da8eb0 None admin] RBAC: Authorizing `identity:get_group(group_id=9980b9236dfe4cdabb8317a18b888217)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:54.944972 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d17e6e-0c0f-45e0-99ff-7706c9da8eb0 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:54.947814 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 203/395] 10.222.0.55 () {56 vars in 1180 bytes} [Fri Sep 23 14:58:54 2022] GET /identity/v3/groups/9980b9236dfe4cdabb8317a18b888217 => generated 226 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:58:54.947941 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f904d8-93d3-4543-b110-b4ee2d0934dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:54.948557 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f904d8-93d3-4543-b110-b4ee2d0934dd None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:54.948837 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f904d8-93d3-4543-b110-b4ee2d0934dd None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:54.949154 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f904d8-93d3-4543-b110-b4ee2d0934dd None admin] PATH_INFO: `/v3/roles` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:54.949920 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f904d8-93d3-4543-b110-b4ee2d0934dd None admin] RBAC: Adding query filter params (name=member) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:58:54.950225 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f904d8-93d3-4543-b110-b4ee2d0934dd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:54.951582 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f904d8-93d3-4543-b110-b4ee2d0934dd None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:54.959049 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d17e6e-0c0f-45e0-99ff-7706c9da8eb0 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:54.975153 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 193/396] 10.222.0.55 () {56 vars in 1101 bytes} [Fri Sep 23 14:58:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 14:58:54.981993 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f904d8-93d3-4543-b110-b4ee2d0934dd None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:55.106845 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:55.108388 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d17e6e-0c0f-45e0-99ff-7706c9da8eb0 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:55.162404 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f904d8-93d3-4543-b110-b4ee2d0934dd None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:55.169294 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00e0594-f26e-4e0b-9d2c-3a224436fd87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:55.170479 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00e0594-f26e-4e0b-9d2c-3a224436fd87 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:55.171045 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00e0594-f26e-4e0b-9d2c-3a224436fd87 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:55.171670 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00e0594-f26e-4e0b-9d2c-3a224436fd87 None admin] PATH_INFO: `/v3/projects/0ffbe3ca86954853be08ecf627d6af00` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:55.174192 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00e0594-f26e-4e0b-9d2c-3a224436fd87 None admin] RBAC: Authorizing `identity:get_project(project_id=0ffbe3ca86954853be08ecf627d6af00)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:55.176268 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00e0594-f26e-4e0b-9d2c-3a224436fd87 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:55.184018 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 204/397] 10.222.0.55 () {56 vars in 1186 bytes} [Fri Sep 23 14:58:54 2022] GET /identity/v3/projects/0ffbe3ca86954853be08ecf627d6af00 => generated 298 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 14:58:55.190475 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00e0594-f26e-4e0b-9d2c-3a224436fd87 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:55.227322 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1eeb075-a39f-48ca-ad5b-771afcf3df06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:55.228616 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1eeb075-a39f-48ca-ad5b-771afcf3df06 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:55.228908 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1eeb075-a39f-48ca-ad5b-771afcf3df06 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:55.229176 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1eeb075-a39f-48ca-ad5b-771afcf3df06 None admin] PATH_INFO: `/v3/groups/9980b9236dfe4cdabb8317a18b888217` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:55.230339 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1eeb075-a39f-48ca-ad5b-771afcf3df06 None admin] RBAC: Authorizing `identity:get_group(group_id=9980b9236dfe4cdabb8317a18b888217)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:55.231843 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1eeb075-a39f-48ca-ad5b-771afcf3df06 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:55.234333 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 194/398] 10.222.0.55 () {56 vars in 1180 bytes} [Fri Sep 23 14:58:54 2022] GET /identity/v3/groups/9980b9236dfe4cdabb8317a18b888217 => generated 226 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 14:58:55.235043 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83aaba8d-8c46-4b0e-a06f-8ef3e7b18ccc None None] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:55.235331 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83aaba8d-8c46-4b0e-a06f-8ef3e7b18ccc None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:55.235654 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83aaba8d-8c46-4b0e-a06f-8ef3e7b18ccc None None] PATH_INFO: `/` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:55.237148 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 195/399] 10.222.0.55 () {54 vars in 927 bytes} [Fri Sep 23 14:58:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 14:58:55.242643 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cba953c-2008-40a2-8327-c098b6b3c0b4 None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:55.243873 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cba953c-2008-40a2-8327-c098b6b3c0b4 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:55.245372 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cba953c-2008-40a2-8327-c098b6b3c0b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:55.257599 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:55.269879 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-8cba953c-2008-40a2-8327-c098b6b3c0b4 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 14:58:55.272187 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-8cba953c-2008-40a2-8327-c098b6b3c0b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 14:58:55.331128 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00e0594-f26e-4e0b-9d2c-3a224436fd87 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:55.359796 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 196/400] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Sep 23 14:58:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 23 14:58:55.360718 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cba953c-2008-40a2-8327-c098b6b3c0b4 None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:55.385039 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8aae6d-e0ee-4d1b-9f60-ec6bca2b2428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:55.386402 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8aae6d-e0ee-4d1b-9f60-ec6bca2b2428 None admin] REQUEST_METHOD: `PUT` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:55.386734 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8aae6d-e0ee-4d1b-9f60-ec6bca2b2428 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:55.387182 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8aae6d-e0ee-4d1b-9f60-ec6bca2b2428 None admin] PATH_INFO: `/v3/projects/0ffbe3ca86954853be08ecf627d6af00/groups/9980b9236dfe4cdabb8317a18b888217/roles/5c74fdc0e9cc44538cd8282006ed4443` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:55.390044 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8aae6d-e0ee-4d1b-9f60-ec6bca2b2428 None admin] RBAC: Authorizing `identity:create_grant(project_id=0ffbe3ca86954853be08ecf627d6af00, group_id=9980b9236dfe4cdabb8317a18b888217, role_id=5c74fdc0e9cc44538cd8282006ed4443)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:55.391900 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8aae6d-e0ee-4d1b-9f60-ec6bca2b2428 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:55.427277 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:55.472263 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 205/401] 10.222.0.55 () {58 vars in 1442 bytes} [Fri Sep 23 14:58:55 2022] PUT /identity/v3/projects/0ffbe3ca86954853be08ecf627d6af00/groups/9980b9236dfe4cdabb8317a18b888217/roles/5c74fdc0e9cc44538cd8282006ed4443 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 14:58:55.473927 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec35bcf-b526-4017-a9d7-d278d72df9ca None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:55.474317 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec35bcf-b526-4017-a9d7-d278d72df9ca None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:55.474692 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec35bcf-b526-4017-a9d7-d278d72df9ca None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:55.476160 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 206/402] 10.222.0.55 () {54 vars in 927 bytes} [Fri Sep 23 14:58:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 14:58:55.481912 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799a94ec-dd1c-4a80-a8d5-a03039650e1f None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:55.482358 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799a94ec-dd1c-4a80-a8d5-a03039650e1f None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:55.482710 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799a94ec-dd1c-4a80-a8d5-a03039650e1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:55.484118 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cba953c-2008-40a2-8327-c098b6b3c0b4 None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:55.524722 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-799a94ec-dd1c-4a80-a8d5-a03039650e1f None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 14:58:55.525864 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-799a94ec-dd1c-4a80-a8d5-a03039650e1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 14:58:55.546535 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e2ee04-81af-4714-9d9b-020b12b4e803 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:55.548177 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e2ee04-81af-4714-9d9b-020b12b4e803 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:55.548815 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e2ee04-81af-4714-9d9b-020b12b4e803 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:55.549283 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e2ee04-81af-4714-9d9b-020b12b4e803 None admin] PATH_INFO: `/v3/projects/0ffbe3ca86954853be08ecf627d6af00` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:55.554674 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e2ee04-81af-4714-9d9b-020b12b4e803 None admin] RBAC: Authorizing `identity:get_project(project_id=0ffbe3ca86954853be08ecf627d6af00)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:55.559811 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e2ee04-81af-4714-9d9b-020b12b4e803 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:55.563963 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 197/403] 10.222.0.55 () {56 vars in 1186 bytes} [Fri Sep 23 14:58:55 2022] GET /identity/v3/projects/0ffbe3ca86954853be08ecf627d6af00 => generated 298 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:58:55.571009 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e2ee04-81af-4714-9d9b-020b12b4e803 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:55.704041 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 207/404] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Sep 23 14:58:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 23 14:58:55.704041 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799a94ec-dd1c-4a80-a8d5-a03039650e1f None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:55.737069 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:55.761911 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e2ee04-81af-4714-9d9b-020b12b4e803 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:55.814796 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:55.835267 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ed3f64-ebac-4373-a812-9f7f985e0a4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:55.836341 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ed3f64-ebac-4373-a812-9f7f985e0a4f None admin] REQUEST_METHOD: `PUT` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:55.836761 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ed3f64-ebac-4373-a812-9f7f985e0a4f None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:55.837756 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ed3f64-ebac-4373-a812-9f7f985e0a4f None admin] PATH_INFO: `/v3/projects/0ffbe3ca86954853be08ecf627d6af00/groups/9980b9236dfe4cdabb8317a18b888217/roles/af257be2dee54d6b942f58650196582c` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:55.841698 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ed3f64-ebac-4373-a812-9f7f985e0a4f None admin] RBAC: Authorizing `identity:create_grant(project_id=0ffbe3ca86954853be08ecf627d6af00, group_id=9980b9236dfe4cdabb8317a18b888217, role_id=af257be2dee54d6b942f58650196582c)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:55.843970 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ed3f64-ebac-4373-a812-9f7f985e0a4f None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:55.869547 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799a94ec-dd1c-4a80-a8d5-a03039650e1f None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:55.908613 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcf5ba3-b880-463a-bbbe-38c34bc47c1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:55.909620 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcf5ba3-b880-463a-bbbe-38c34bc47c1c None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:55.910035 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcf5ba3-b880-463a-bbbe-38c34bc47c1c None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:55.910512 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcf5ba3-b880-463a-bbbe-38c34bc47c1c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:55.922709 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dcf5ba3-b880-463a-bbbe-38c34bc47c1c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 14:58:55.928284 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-6dcf5ba3-b880-463a-bbbe-38c34bc47c1c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 23 14:58:55.928284 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 14:58:55.928284 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 14:58:55.928284 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 14:58:55.928284 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 14:58:55.928284 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 14:58:55.928284 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 14:58:55.928284 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 14:58:55.928284 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 14:58:55.928284 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 14:58:55.928284 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 14:58:55.928284 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 14:58:55.928284 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 14:58:55.928284 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 14:58:55.928284 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 14:58:55.928284 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 14:58:55.929287 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 14:58:55.929287 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 14:58:55.929287 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 14:58:55.929287 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 14:58:55.929287 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 14:58:55.929287 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 14:58:55.929287 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 14:58:55.929287 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 14:58:55.929287 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 14:58:55.929287 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 14:58:55.929287 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 14:58:55.929287 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Sep 23 14:58:55.929287 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 14:58:55.929287 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Sep 23 14:58:55.929287 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 14:58:55.929287 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 14:58:55.930116 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 14:58:55.930116 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Sep 23 14:58:55.930116 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 14:58:55.930116 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Sep 23 14:58:55.930116 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 14:58:55.930116 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 14:58:55.930116 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 14:58:55.930116 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 14:58:55.930116 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 14:58:55.930116 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 14:58:55.930116 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 23 14:58:55.930116 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Sep 23 14:58:55.930808 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 208/405] 10.222.0.55 () {56 vars in 1096 bytes} [Fri Sep 23 14:58:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 14:58:55.937389 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcf5ba3-b880-463a-bbbe-38c34bc47c1c None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:55.947544 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 198/406] 10.222.0.55 () {58 vars in 1442 bytes} [Fri Sep 23 14:58:55 2022] PUT /identity/v3/projects/0ffbe3ca86954853be08ecf627d6af00/groups/9980b9236dfe4cdabb8317a18b888217/roles/af257be2dee54d6b942f58650196582c => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 14:58:56.024542 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ed3f64-ebac-4373-a812-9f7f985e0a4f None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:56.132190 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:56.149185 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcf5ba3-b880-463a-bbbe-38c34bc47c1c None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:56.187932 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13768a0-d8eb-4ab1-960f-71dc33b3935e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:56.188585 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13768a0-d8eb-4ab1-960f-71dc33b3935e None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:56.188873 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13768a0-d8eb-4ab1-960f-71dc33b3935e None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:56.189151 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13768a0-d8eb-4ab1-960f-71dc33b3935e None admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:56.192956 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13768a0-d8eb-4ab1-960f-71dc33b3935e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:58:56.192956 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13768a0-d8eb-4ab1-960f-71dc33b3935e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:56.192956 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13768a0-d8eb-4ab1-960f-71dc33b3935e None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:56.201452 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 209/407] 10.222.0.55 () {56 vars in 1099 bytes} [Fri Sep 23 14:58:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:58:56.208038 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13768a0-d8eb-4ab1-960f-71dc33b3935e None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:56.246020 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ed3f64-ebac-4373-a812-9f7f985e0a4f None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:56.295282 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73dcc2f-7ecd-4465-afdb-060b9b9b0d90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:56.295844 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73dcc2f-7ecd-4465-afdb-060b9b9b0d90 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:56.296173 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73dcc2f-7ecd-4465-afdb-060b9b9b0d90 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:56.296511 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73dcc2f-7ecd-4465-afdb-060b9b9b0d90 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:56.312204 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73dcc2f-7ecd-4465-afdb-060b9b9b0d90 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 14:58:56.319492 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application [None req-d73dcc2f-7ecd-4465-afdb-060b9b9b0d90 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 23 14:58:56.319492 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 14:58:56.319492 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 14:58:56.319492 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 14:58:56.319492 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 14:58:56.319492 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 14:58:56.319492 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 14:58:56.319492 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 14:58:56.319492 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 14:58:56.319492 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 14:58:56.319492 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 14:58:56.319492 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 14:58:56.319492 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 14:58:56.319492 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 14:58:56.319492 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 14:58:56.319492 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 14:58:56.320390 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 14:58:56.320390 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 14:58:56.320390 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 14:58:56.320390 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 14:58:56.320390 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 14:58:56.320390 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 14:58:56.320390 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 14:58:56.320390 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 14:58:56.320390 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 14:58:56.320390 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 14:58:56.320390 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 14:58:56.320390 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application with Lock( Sep 23 14:58:56.320390 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 14:58:56.320390 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._enter() Sep 23 14:58:56.320390 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 14:58:56.320390 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 14:58:56.322068 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 14:58:56.322068 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.creator() Sep 23 14:58:56.322068 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 14:58:56.322068 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application created_value = creator( Sep 23 14:58:56.322068 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 14:58:56.322068 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 14:58:56.322068 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 14:58:56.322068 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 14:58:56.322068 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 14:58:56.322068 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 14:58:56.322068 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 23 14:58:56.322068 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Sep 23 14:58:56.322068 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 199/408] 10.222.0.55 () {56 vars in 1114 bytes} [Fri Sep 23 14:58:56 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 23 14:58:56.329242 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73dcc2f-7ecd-4465-afdb-060b9b9b0d90 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:56.357901 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13768a0-d8eb-4ab1-960f-71dc33b3935e None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:56.403097 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:56.437955 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84db505d-505b-4a52-adba-b298ea080bc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:56.438606 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84db505d-505b-4a52-adba-b298ea080bc1 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:56.438887 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84db505d-505b-4a52-adba-b298ea080bc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:56.439163 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84db505d-505b-4a52-adba-b298ea080bc1 None admin] PATH_INFO: `/v3/groups/0f32957856bc433188452103c17b1175` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:56.441395 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84db505d-505b-4a52-adba-b298ea080bc1 None admin] RBAC: Authorizing `identity:get_group(group_id=0f32957856bc433188452103c17b1175)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:56.442849 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84db505d-505b-4a52-adba-b298ea080bc1 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:56.444479 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 210/409] 10.222.0.55 () {56 vars in 1180 bytes} [Fri Sep 23 14:58:56 2022] GET /identity/v3/groups/0f32957856bc433188452103c17b1175 => generated 229 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 14:58:56.453110 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84db505d-505b-4a52-adba-b298ea080bc1 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:56.489139 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:56.512340 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73dcc2f-7ecd-4465-afdb-060b9b9b0d90 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:56.567028 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8720b2-3150-4099-96fe-bb52e61bd25d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:56.567566 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8720b2-3150-4099-96fe-bb52e61bd25d None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:56.567843 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8720b2-3150-4099-96fe-bb52e61bd25d None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:56.568147 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8720b2-3150-4099-96fe-bb52e61bd25d None admin] PATH_INFO: `/v3/roles` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:56.569357 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8720b2-3150-4099-96fe-bb52e61bd25d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:58:56.569680 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8720b2-3150-4099-96fe-bb52e61bd25d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:56.571303 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8720b2-3150-4099-96fe-bb52e61bd25d None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:56.582056 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 200/410] 10.222.0.55 () {56 vars in 1111 bytes} [Fri Sep 23 14:58:56 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:58:56.587414 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8720b2-3150-4099-96fe-bb52e61bd25d None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:56.668746 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84db505d-505b-4a52-adba-b298ea080bc1 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:56.710481 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bb702b-e59c-4544-9b99-5aeb88637b7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:56.711136 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bb702b-e59c-4544-9b99-5aeb88637b7c None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:56.711420 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bb702b-e59c-4544-9b99-5aeb88637b7c None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:56.711699 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bb702b-e59c-4544-9b99-5aeb88637b7c None admin] PATH_INFO: `/v3/projects/23b5c58d198d4456bfeec0a8daae1113` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:56.716710 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bb702b-e59c-4544-9b99-5aeb88637b7c None admin] RBAC: Authorizing `identity:get_project(project_id=23b5c58d198d4456bfeec0a8daae1113)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:56.718901 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bb702b-e59c-4544-9b99-5aeb88637b7c None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:56.720893 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 211/411] 10.222.0.55 () {56 vars in 1186 bytes} [Fri Sep 23 14:58:56 2022] GET /identity/v3/projects/23b5c58d198d4456bfeec0a8daae1113 => generated 344 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:58:56.726537 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bb702b-e59c-4544-9b99-5aeb88637b7c None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:56.809916 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8720b2-3150-4099-96fe-bb52e61bd25d None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:56.825553 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:56.827573 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:56.845761 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bb702b-e59c-4544-9b99-5aeb88637b7c None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:56.882641 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63044317-48d2-4f30-9e38-c3ab4fcec615 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:56.883552 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63044317-48d2-4f30-9e38-c3ab4fcec615 None admin] REQUEST_METHOD: `PUT` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:56.884794 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63044317-48d2-4f30-9e38-c3ab4fcec615 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:56.885297 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63044317-48d2-4f30-9e38-c3ab4fcec615 None admin] PATH_INFO: `/v3/projects/23b5c58d198d4456bfeec0a8daae1113/groups/0f32957856bc433188452103c17b1175/roles/8e2919e79df040e8926f21607818b58c` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:56.887487 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63044317-48d2-4f30-9e38-c3ab4fcec615 None admin] RBAC: Authorizing `identity:create_grant(project_id=23b5c58d198d4456bfeec0a8daae1113, group_id=0f32957856bc433188452103c17b1175, role_id=8e2919e79df040e8926f21607818b58c)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:56.889158 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63044317-48d2-4f30-9e38-c3ab4fcec615 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:56.892680 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af1642a-8a89-474b-b215-272935a5ef07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:56.893359 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af1642a-8a89-474b-b215-272935a5ef07 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:56.893706 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af1642a-8a89-474b-b215-272935a5ef07 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:56.893979 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af1642a-8a89-474b-b215-272935a5ef07 None admin] PATH_INFO: `/v3/groups/9980b9236dfe4cdabb8317a18b888217` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:56.895283 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af1642a-8a89-474b-b215-272935a5ef07 None admin] RBAC: Authorizing `identity:get_group(group_id=9980b9236dfe4cdabb8317a18b888217)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:56.897396 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af1642a-8a89-474b-b215-272935a5ef07 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:56.899759 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 201/412] 10.222.0.55 () {56 vars in 1180 bytes} [Fri Sep 23 14:58:56 2022] GET /identity/v3/groups/9980b9236dfe4cdabb8317a18b888217 => generated 226 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:58:56.908186 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af1642a-8a89-474b-b215-272935a5ef07 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:57.002661 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 212/413] 10.222.0.55 () {58 vars in 1442 bytes} [Fri Sep 23 14:58:56 2022] PUT /identity/v3/projects/23b5c58d198d4456bfeec0a8daae1113/groups/0f32957856bc433188452103c17b1175/roles/8e2919e79df040e8926f21607818b58c => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 14:58:57.043868 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af1642a-8a89-474b-b215-272935a5ef07 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:57.073679 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d6d54a-c78e-40f3-901d-7e985f4447c9 None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:57.074105 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d6d54a-c78e-40f3-901d-7e985f4447c9 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:57.074335 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d6d54a-c78e-40f3-901d-7e985f4447c9 None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:57.080934 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 213/414] 10.222.0.55 () {54 vars in 927 bytes} [Fri Sep 23 14:58:57 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 14:58:57.086477 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b01f60f-7e60-4f37-9cdc-b7afead3a3d0 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:57.086751 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b01f60f-7e60-4f37-9cdc-b7afead3a3d0 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:57.087163 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b01f60f-7e60-4f37-9cdc-b7afead3a3d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:57.089710 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acffd70-9555-49e6-98f2-a235bacfb347 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:57.090617 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acffd70-9555-49e6-98f2-a235bacfb347 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:57.091291 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acffd70-9555-49e6-98f2-a235bacfb347 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:57.091812 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acffd70-9555-49e6-98f2-a235bacfb347 None admin] PATH_INFO: `/v3/projects/346756b911774d55af2a9ad3de2143b4` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:57.093321 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acffd70-9555-49e6-98f2-a235bacfb347 None admin] RBAC: Authorizing `identity:get_project(project_id=346756b911774d55af2a9ad3de2143b4)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:57.095048 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acffd70-9555-49e6-98f2-a235bacfb347 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:57.097057 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 202/415] 10.222.0.55 () {56 vars in 1186 bytes} [Fri Sep 23 14:58:56 2022] GET /identity/v3/projects/346756b911774d55af2a9ad3de2143b4 => generated 302 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:58:57.104053 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acffd70-9555-49e6-98f2-a235bacfb347 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:57.107871 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-0b01f60f-7e60-4f37-9cdc-b7afead3a3d0 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 14:58:57.109346 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-0b01f60f-7e60-4f37-9cdc-b7afead3a3d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 14:58:57.166257 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:57.226942 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 214/416] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Sep 23 14:58:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 23 14:58:57.258308 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acffd70-9555-49e6-98f2-a235bacfb347 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:57.295860 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d499b223-4208-40ef-b664-c66229b26142 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:57.296573 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d499b223-4208-40ef-b664-c66229b26142 None admin] REQUEST_METHOD: `PUT` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:57.296857 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d499b223-4208-40ef-b664-c66229b26142 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:57.297138 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d499b223-4208-40ef-b664-c66229b26142 None admin] PATH_INFO: `/v3/projects/346756b911774d55af2a9ad3de2143b4/groups/9980b9236dfe4cdabb8317a18b888217/roles/5c74fdc0e9cc44538cd8282006ed4443` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:57.299395 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d499b223-4208-40ef-b664-c66229b26142 None admin] RBAC: Authorizing `identity:create_grant(project_id=346756b911774d55af2a9ad3de2143b4, group_id=9980b9236dfe4cdabb8317a18b888217, role_id=5c74fdc0e9cc44538cd8282006ed4443)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:57.300946 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d499b223-4208-40ef-b664-c66229b26142 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:57.409564 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 203/417] 10.222.0.55 () {58 vars in 1442 bytes} [Fri Sep 23 14:58:57 2022] PUT /identity/v3/projects/346756b911774d55af2a9ad3de2143b4/groups/9980b9236dfe4cdabb8317a18b888217/roles/5c74fdc0e9cc44538cd8282006ed4443 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 14:58:57.549416 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b01f60f-7e60-4f37-9cdc-b7afead3a3d0 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:57.650520 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:57.691932 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b01f60f-7e60-4f37-9cdc-b7afead3a3d0 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:57.740750 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a963c4-b0ff-4c1d-9abb-56a94581e4c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:57.741403 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a963c4-b0ff-4c1d-9abb-56a94581e4c0 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:57.741746 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a963c4-b0ff-4c1d-9abb-56a94581e4c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:57.742016 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a963c4-b0ff-4c1d-9abb-56a94581e4c0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:57.755676 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a963c4-b0ff-4c1d-9abb-56a94581e4c0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 14:58:57.760533 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-14a963c4-b0ff-4c1d-9abb-56a94581e4c0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 23 14:58:57.760533 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 14:58:57.760533 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 14:58:57.760533 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 14:58:57.760533 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 14:58:57.760533 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 14:58:57.760533 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 14:58:57.760533 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 14:58:57.760533 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 14:58:57.760533 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 14:58:57.760533 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 14:58:57.760533 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 14:58:57.760533 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 14:58:57.760533 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 14:58:57.760533 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 14:58:57.760533 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 14:58:57.761394 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 14:58:57.761394 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 14:58:57.761394 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 14:58:57.761394 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 14:58:57.761394 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 14:58:57.761394 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 14:58:57.761394 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 14:58:57.761394 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 14:58:57.761394 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 14:58:57.761394 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 14:58:57.761394 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 14:58:57.761394 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Sep 23 14:58:57.761394 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 14:58:57.761394 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Sep 23 14:58:57.761394 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 14:58:57.761394 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 14:58:57.762184 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 14:58:57.762184 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Sep 23 14:58:57.762184 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 14:58:57.762184 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Sep 23 14:58:57.762184 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 14:58:57.762184 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 14:58:57.762184 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 14:58:57.762184 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 14:58:57.762184 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 14:58:57.762184 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 14:58:57.762184 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 23 14:58:57.762184 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Sep 23 14:58:57.762184 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 215/418] 10.222.0.55 () {56 vars in 1099 bytes} [Fri Sep 23 14:58:57 2022] GET /identity/v3/roles/member => generated 84 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 14:58:57.767694 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d499b223-4208-40ef-b664-c66229b26142 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:57.885742 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d499b223-4208-40ef-b664-c66229b26142 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:57.894427 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:57.927785 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e02feae-9aba-494e-a652-30fed2fe9e8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:57.928454 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e02feae-9aba-494e-a652-30fed2fe9e8d None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:57.928735 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e02feae-9aba-494e-a652-30fed2fe9e8d None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:57.929012 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e02feae-9aba-494e-a652-30fed2fe9e8d None admin] PATH_INFO: `/v3/roles` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:57.929696 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e02feae-9aba-494e-a652-30fed2fe9e8d None admin] RBAC: Adding query filter params (name=member) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:58:57.930004 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e02feae-9aba-494e-a652-30fed2fe9e8d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:57.931932 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e02feae-9aba-494e-a652-30fed2fe9e8d None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:57.942007 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 204/419] 10.222.0.55 () {56 vars in 1101 bytes} [Fri Sep 23 14:58:57 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:58:57.947880 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a963c4-b0ff-4c1d-9abb-56a94581e4c0 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:58.103452 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a963c4-b0ff-4c1d-9abb-56a94581e4c0 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:58.150309 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8624891e-b0a2-429d-a3d5-53717cb135ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:58.150929 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8624891e-b0a2-429d-a3d5-53717cb135ba None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:58.151208 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8624891e-b0a2-429d-a3d5-53717cb135ba None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:58.151478 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8624891e-b0a2-429d-a3d5-53717cb135ba None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:58.158290 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:58.184951 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae2f979-996f-4a32-8c23-27c172ce6a56 None None] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:58.185262 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae2f979-996f-4a32-8c23-27c172ce6a56 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:58.185497 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae2f979-996f-4a32-8c23-27c172ce6a56 None None] PATH_INFO: `/` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:58.187098 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 205/420] 10.222.0.55 () {54 vars in 927 bytes} [Fri Sep 23 14:58:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 14:58:58.193198 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af216a9c-0c87-4be1-b465-ec9e3fdc33be None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:58.193419 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af216a9c-0c87-4be1-b465-ec9e3fdc33be None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:58.193694 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af216a9c-0c87-4be1-b465-ec9e3fdc33be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:58.221465 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-af216a9c-0c87-4be1-b465-ec9e3fdc33be None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 14:58:58.222672 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-af216a9c-0c87-4be1-b465-ec9e3fdc33be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 14:58:58.282435 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8624891e-b0a2-429d-a3d5-53717cb135ba None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:58.284280 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8624891e-b0a2-429d-a3d5-53717cb135ba None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:58.314666 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 206/421] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Sep 23 14:58:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 23 14:58:58.472841 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 216/422] 10.222.0.55 () {58 vars in 1282 bytes} [Fri Sep 23 14:58:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 526 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 23 14:58:58.479388 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af216a9c-0c87-4be1-b465-ec9e3fdc33be None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:58.604100 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:58.607915 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af216a9c-0c87-4be1-b465-ec9e3fdc33be None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:58.643607 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e577eec8-1914-4f7d-834a-5a17447a17db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:58.644276 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e577eec8-1914-4f7d-834a-5a17447a17db None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:58.644553 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e577eec8-1914-4f7d-834a-5a17447a17db None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:58.644827 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e577eec8-1914-4f7d-834a-5a17447a17db None admin] PATH_INFO: `/v3/projects/346756b911774d55af2a9ad3de2143b4` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:58.645944 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e577eec8-1914-4f7d-834a-5a17447a17db None admin] RBAC: Authorizing `identity:get_project(project_id=346756b911774d55af2a9ad3de2143b4)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:58.647408 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e577eec8-1914-4f7d-834a-5a17447a17db None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:58.649511 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 207/423] 10.222.0.55 () {56 vars in 1186 bytes} [Fri Sep 23 14:58:58 2022] GET /identity/v3/projects/346756b911774d55af2a9ad3de2143b4 => generated 302 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:58:58.655016 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8624891e-b0a2-429d-a3d5-53717cb135ba None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:58.690509 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:58.740604 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e577eec8-1914-4f7d-834a-5a17447a17db None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:58.815578 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8624891e-b0a2-429d-a3d5-53717cb135ba None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:58.875586 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e577eec8-1914-4f7d-834a-5a17447a17db None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:58.897590 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a41ee61-197d-48b7-978d-0c8b7078f4fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:58.898468 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a41ee61-197d-48b7-978d-0c8b7078f4fe None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:58.898748 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a41ee61-197d-48b7-978d-0c8b7078f4fe None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:58.899020 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a41ee61-197d-48b7-978d-0c8b7078f4fe None admin] PATH_INFO: `/v3/groups/9980b9236dfe4cdabb8317a18b888217` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:58.900284 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a41ee61-197d-48b7-978d-0c8b7078f4fe None admin] RBAC: Authorizing `identity:get_group(group_id=9980b9236dfe4cdabb8317a18b888217)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:58.902142 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a41ee61-197d-48b7-978d-0c8b7078f4fe None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:58.904104 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 217/424] 10.222.0.55 () {56 vars in 1180 bytes} [Fri Sep 23 14:58:58 2022] GET /identity/v3/groups/9980b9236dfe4cdabb8317a18b888217 => generated 226 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:58:58.905336 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4e5be0-a518-4ef7-a1d2-11af3a91d600 None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:58.905621 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4e5be0-a518-4ef7-a1d2-11af3a91d600 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:58.905894 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4e5be0-a518-4ef7-a1d2-11af3a91d600 None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:58.911507 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 218/425] 10.222.0.55 () {54 vars in 927 bytes} [Fri Sep 23 14:58:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 14:58:58.912394 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4e5be0-a518-4ef7-a1d2-11af3a91d600 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:58.919538 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ccee3a-bc09-4710-953c-8cd443ee4267 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:58.920216 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ccee3a-bc09-4710-953c-8cd443ee4267 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:58.920497 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ccee3a-bc09-4710-953c-8cd443ee4267 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:58.920826 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ccee3a-bc09-4710-953c-8cd443ee4267 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:58.938579 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ccee3a-bc09-4710-953c-8cd443ee4267 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 14:58:58.943663 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application [None req-97ccee3a-bc09-4710-953c-8cd443ee4267 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 23 14:58:58.943663 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 14:58:58.943663 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 14:58:58.943663 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 14:58:58.943663 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 14:58:58.943663 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 14:58:58.943663 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 14:58:58.943663 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 14:58:58.943663 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 14:58:58.943663 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 14:58:58.943663 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 14:58:58.943663 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 14:58:58.943663 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 14:58:58.943663 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 14:58:58.943663 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 14:58:58.943663 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 14:58:58.944787 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 14:58:58.944787 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 14:58:58.944787 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 14:58:58.944787 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 14:58:58.944787 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 14:58:58.944787 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 14:58:58.944787 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 14:58:58.944787 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 14:58:58.944787 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 14:58:58.944787 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 14:58:58.944787 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 14:58:58.944787 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application with Lock( Sep 23 14:58:58.944787 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 14:58:58.944787 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._enter() Sep 23 14:58:58.944787 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 14:58:58.944787 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 14:58:58.945583 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 14:58:58.945583 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.creator() Sep 23 14:58:58.945583 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 14:58:58.945583 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application created_value = creator( Sep 23 14:58:58.945583 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 14:58:58.945583 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 14:58:58.945583 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 14:58:58.945583 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 14:58:58.945583 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 14:58:58.945583 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 14:58:58.945583 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 23 14:58:58.945583 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Sep 23 14:58:58.945583 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 208/426] 10.222.0.55 () {56 vars in 1114 bytes} [Fri Sep 23 14:58:58 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 23 14:58:58.946282 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddb29b0-9730-43ab-be2c-9a850c0e899b None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:58.946777 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddb29b0-9730-43ab-be2c-9a850c0e899b None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:58.947413 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddb29b0-9730-43ab-be2c-9a850c0e899b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:58.968032 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-eddb29b0-9730-43ab-be2c-9a850c0e899b None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 14:58:58.969820 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-eddb29b0-9730-43ab-be2c-9a850c0e899b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 14:58:58.999535 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:59.046710 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4e5be0-a518-4ef7-a1d2-11af3a91d600 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:59.049003 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:59.064113 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 209/427] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Sep 23 14:58:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 14:58:59.065319 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddb29b0-9730-43ab-be2c-9a850c0e899b None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:59.112185 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4235cef0-1f8f-4b31-936e-933bb9b60669 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:59.113904 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4235cef0-1f8f-4b31-936e-933bb9b60669 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:59.114184 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4235cef0-1f8f-4b31-936e-933bb9b60669 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:59.114460 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4235cef0-1f8f-4b31-936e-933bb9b60669 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:59.115187 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4235cef0-1f8f-4b31-936e-933bb9b60669 None admin] RBAC: Adding query filter params (group.id=9980b9236dfe4cdabb8317a18b888217, role.id=af257be2dee54d6b942f58650196582c, scope.project.id=346756b911774d55af2a9ad3de2143b4) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:58:59.115496 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4235cef0-1f8f-4b31-936e-933bb9b60669 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:59.117343 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4235cef0-1f8f-4b31-936e-933bb9b60669 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:59.136007 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 219/428] 10.222.0.55 () {56 vars in 1376 bytes} [Fri Sep 23 14:58:58 2022] GET /identity/v3/role_assignments?group.id=9980b9236dfe4cdabb8317a18b888217&scope.project.id=346756b911774d55af2a9ad3de2143b4&role.id=af257be2dee54d6b942f58650196582c => generated 611 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:58:59.228713 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddb29b0-9730-43ab-be2c-9a850c0e899b None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:59.267109 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acad2f3-ceae-478a-9391-2b05a1e901b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:59.267645 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acad2f3-ceae-478a-9391-2b05a1e901b1 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:59.267925 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acad2f3-ceae-478a-9391-2b05a1e901b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:59.268228 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acad2f3-ceae-478a-9391-2b05a1e901b1 None admin] PATH_INFO: `/v3/roles` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:59.268915 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acad2f3-ceae-478a-9391-2b05a1e901b1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:58:59.269223 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acad2f3-ceae-478a-9391-2b05a1e901b1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:59.270650 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acad2f3-ceae-478a-9391-2b05a1e901b1 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:59.281047 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 210/429] 10.222.0.55 () {56 vars in 1111 bytes} [Fri Sep 23 14:58:59 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:58:59.286961 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4235cef0-1f8f-4b31-936e-933bb9b60669 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:59.401328 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4235cef0-1f8f-4b31-936e-933bb9b60669 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:59.440333 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf36a03-2934-4810-ae22-38203c7b93c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:59.440958 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf36a03-2934-4810-ae22-38203c7b93c2 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:59.441239 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf36a03-2934-4810-ae22-38203c7b93c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:59.441514 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf36a03-2934-4810-ae22-38203c7b93c2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:59.484179 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:59.523070 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf36a03-2934-4810-ae22-38203c7b93c2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:59.524529 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf36a03-2934-4810-ae22-38203c7b93c2 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:59.533003 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acad2f3-ceae-478a-9391-2b05a1e901b1 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:59.559572 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:59.666561 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 220/430] 10.222.0.55 () {58 vars in 1282 bytes} [Fri Sep 23 14:58:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 23 14:58:59.672229 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70c97f0-1588-4f58-aacb-838753a4350e None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:59.672630 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70c97f0-1588-4f58-aacb-838753a4350e None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:59.672813 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acad2f3-ceae-478a-9391-2b05a1e901b1 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:59.673049 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70c97f0-1588-4f58-aacb-838753a4350e None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:59.674758 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 221/431] 10.222.0.55 () {54 vars in 927 bytes} [Fri Sep 23 14:58:59 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 14:58:59.675575 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70c97f0-1588-4f58-aacb-838753a4350e None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:59.711500 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8804c1c-3379-49c7-a81e-4a784c722e32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:59.712719 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8804c1c-3379-49c7-a81e-4a784c722e32 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:59.713008 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8804c1c-3379-49c7-a81e-4a784c722e32 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:59.713281 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8804c1c-3379-49c7-a81e-4a784c722e32 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:59.724748 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8804c1c-3379-49c7-a81e-4a784c722e32 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 14:58:59.731461 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application [None req-e8804c1c-3379-49c7-a81e-4a784c722e32 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 23 14:58:59.731461 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 14:58:59.731461 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 14:58:59.731461 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 14:58:59.731461 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 14:58:59.731461 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 14:58:59.731461 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 14:58:59.731461 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 14:58:59.731461 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 14:58:59.731461 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 14:58:59.731461 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 14:58:59.731461 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 14:58:59.731461 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 14:58:59.731461 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 14:58:59.731461 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 14:58:59.731461 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 14:58:59.732399 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 14:58:59.732399 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 14:58:59.732399 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 14:58:59.732399 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 14:58:59.732399 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 14:58:59.732399 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 14:58:59.732399 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 14:58:59.732399 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 14:58:59.732399 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 14:58:59.732399 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 14:58:59.732399 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 14:58:59.732399 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application with Lock( Sep 23 14:58:59.732399 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 14:58:59.732399 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._enter() Sep 23 14:58:59.732399 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 14:58:59.732399 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 14:58:59.738153 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 14:58:59.738153 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.creator() Sep 23 14:58:59.738153 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 14:58:59.738153 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application created_value = creator( Sep 23 14:58:59.738153 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 14:58:59.738153 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 14:58:59.738153 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 14:58:59.738153 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 14:58:59.738153 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 14:58:59.738153 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 14:58:59.738153 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 23 14:58:59.738153 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Sep 23 14:58:59.738153 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 211/432] 10.222.0.55 () {56 vars in 1099 bytes} [Fri Sep 23 14:58:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 14:58:59.738153 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421accd1-8508-42ab-a5c0-4eab75b8e344 None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:59.738873 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421accd1-8508-42ab-a5c0-4eab75b8e344 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:59.738873 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421accd1-8508-42ab-a5c0-4eab75b8e344 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:59.758086 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-421accd1-8508-42ab-a5c0-4eab75b8e344 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 14:58:59.760881 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-421accd1-8508-42ab-a5c0-4eab75b8e344 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 14:58:59.763201 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:59.800610 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70c97f0-1588-4f58-aacb-838753a4350e None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:58:59.839051 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79058fa7-7411-42af-a4b1-ffed4cb54bcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:58:59.839716 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79058fa7-7411-42af-a4b1-ffed4cb54bcb None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:58:59.840027 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79058fa7-7411-42af-a4b1-ffed4cb54bcb None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:58:59.840474 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79058fa7-7411-42af-a4b1-ffed4cb54bcb None admin] PATH_INFO: `/v3/projects/0ffbe3ca86954853be08ecf627d6af00` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:58:59.841451 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79058fa7-7411-42af-a4b1-ffed4cb54bcb None admin] RBAC: Authorizing `identity:get_project(project_id=0ffbe3ca86954853be08ecf627d6af00)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:58:59.842885 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79058fa7-7411-42af-a4b1-ffed4cb54bcb None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:58:59.848850 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 222/433] 10.222.0.55 () {56 vars in 1186 bytes} [Fri Sep 23 14:58:59 2022] GET /identity/v3/projects/0ffbe3ca86954853be08ecf627d6af00 => generated 298 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:58:59.848850 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79058fa7-7411-42af-a4b1-ffed4cb54bcb None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:59.853072 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 212/434] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Sep 23 14:58:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 14:58:59.860033 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421accd1-8508-42ab-a5c0-4eab75b8e344 None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:58:59.870479 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:58:59.973736 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79058fa7-7411-42af-a4b1-ffed4cb54bcb None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:00.004917 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421accd1-8508-42ab-a5c0-4eab75b8e344 None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:00.012132 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1d21f4-9774-4d79-b157-e4f3f7ea19ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:00.012774 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1d21f4-9774-4d79-b157-e4f3f7ea19ac None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:00.013053 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1d21f4-9774-4d79-b157-e4f3f7ea19ac None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:00.013325 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1d21f4-9774-4d79-b157-e4f3f7ea19ac None admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:00.013987 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1d21f4-9774-4d79-b157-e4f3f7ea19ac None admin] RBAC: Adding query filter params (name=member) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:59:00.014288 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1d21f4-9774-4d79-b157-e4f3f7ea19ac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:00.015752 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1d21f4-9774-4d79-b157-e4f3f7ea19ac None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:00.017429 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:00.026807 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 223/435] 10.222.0.55 () {56 vars in 1101 bytes} [Fri Sep 23 14:58:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:59:00.032735 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1d21f4-9774-4d79-b157-e4f3f7ea19ac None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:00.054058 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0719049-9ca4-4fd0-b13f-35eca5fcacab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:00.054698 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0719049-9ca4-4fd0-b13f-35eca5fcacab None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:00.054971 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0719049-9ca4-4fd0-b13f-35eca5fcacab None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:00.055236 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0719049-9ca4-4fd0-b13f-35eca5fcacab None admin] PATH_INFO: `/v3/groups/9980b9236dfe4cdabb8317a18b888217` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:00.056655 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0719049-9ca4-4fd0-b13f-35eca5fcacab None admin] RBAC: Authorizing `identity:get_group(group_id=9980b9236dfe4cdabb8317a18b888217)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:00.058276 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0719049-9ca4-4fd0-b13f-35eca5fcacab None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:00.060391 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 213/436] 10.222.0.55 () {56 vars in 1180 bytes} [Fri Sep 23 14:58:59 2022] GET /identity/v3/groups/9980b9236dfe4cdabb8317a18b888217 => generated 226 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:59:00.066245 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0719049-9ca4-4fd0-b13f-35eca5fcacab None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:00.152132 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:00.176125 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0719049-9ca4-4fd0-b13f-35eca5fcacab None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:00.178646 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1d21f4-9774-4d79-b157-e4f3f7ea19ac None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:00.213760 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd046ef-9a04-4b7d-9c50-399e0ae4cfab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:00.214218 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a06bbb-68ee-4640-a8ad-b93c5177ac98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:00.214456 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd046ef-9a04-4b7d-9c50-399e0ae4cfab None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:00.214628 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd046ef-9a04-4b7d-9c50-399e0ae4cfab None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:00.214828 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a06bbb-68ee-4640-a8ad-b93c5177ac98 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:00.214918 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd046ef-9a04-4b7d-9c50-399e0ae4cfab None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:00.215112 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a06bbb-68ee-4640-a8ad-b93c5177ac98 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:00.215392 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a06bbb-68ee-4640-a8ad-b93c5177ac98 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:00.215594 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd046ef-9a04-4b7d-9c50-399e0ae4cfab None admin] RBAC: Adding query filter params (group.id=9980b9236dfe4cdabb8317a18b888217, role.id=5c74fdc0e9cc44538cd8282006ed4443, scope.project.id=0ffbe3ca86954853be08ecf627d6af00) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:59:00.215897 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd046ef-9a04-4b7d-9c50-399e0ae4cfab None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:00.217345 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd046ef-9a04-4b7d-9c50-399e0ae4cfab None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:00.222947 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 214/437] 10.222.0.55 () {56 vars in 1376 bytes} [Fri Sep 23 14:59:00 2022] GET /identity/v3/role_assignments?group.id=9980b9236dfe4cdabb8317a18b888217&scope.project.id=0ffbe3ca86954853be08ecf627d6af00&role.id=5c74fdc0e9cc44538cd8282006ed4443 => generated 611 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 14:59:00.223967 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd046ef-9a04-4b7d-9c50-399e0ae4cfab None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:00.285340 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:00.293284 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a06bbb-68ee-4640-a8ad-b93c5177ac98 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:00.294807 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a06bbb-68ee-4640-a8ad-b93c5177ac98 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:00.342187 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd046ef-9a04-4b7d-9c50-399e0ae4cfab None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:00.381072 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf5c901-eec2-48c0-80bb-3cf7a348a91c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:00.381724 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf5c901-eec2-48c0-80bb-3cf7a348a91c None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:00.382004 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf5c901-eec2-48c0-80bb-3cf7a348a91c None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:00.382279 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf5c901-eec2-48c0-80bb-3cf7a348a91c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:00.391706 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:00.395428 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf5c901-eec2-48c0-80bb-3cf7a348a91c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 14:59:00.401257 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application [None req-4bf5c901-eec2-48c0-80bb-3cf7a348a91c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 23 14:59:00.401257 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 14:59:00.401257 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 14:59:00.401257 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 14:59:00.401257 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 14:59:00.401257 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 14:59:00.401257 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 14:59:00.401257 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 14:59:00.401257 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 14:59:00.401257 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 14:59:00.401257 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 14:59:00.401257 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 14:59:00.401257 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 14:59:00.401257 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 14:59:00.401257 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 14:59:00.401257 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 14:59:00.402317 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 14:59:00.402317 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 14:59:00.402317 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 14:59:00.402317 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 14:59:00.402317 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 14:59:00.402317 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 14:59:00.402317 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 14:59:00.402317 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 14:59:00.402317 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 14:59:00.402317 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 14:59:00.402317 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 14:59:00.402317 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application with Lock( Sep 23 14:59:00.402317 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 14:59:00.402317 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._enter() Sep 23 14:59:00.402317 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 14:59:00.402317 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 14:59:00.403110 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 14:59:00.403110 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.creator() Sep 23 14:59:00.403110 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 14:59:00.403110 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application created_value = creator( Sep 23 14:59:00.403110 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 14:59:00.403110 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 14:59:00.403110 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 14:59:00.403110 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 14:59:00.403110 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 14:59:00.403110 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 14:59:00.403110 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 23 14:59:00.403110 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Sep 23 14:59:00.403110 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 215/438] 10.222.0.55 () {56 vars in 1096 bytes} [Fri Sep 23 14:59:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 14:59:00.404473 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d5e4b0-3be0-4a29-a5d1-8fa65b60bb88 None None] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:00.408985 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d5e4b0-3be0-4a29-a5d1-8fa65b60bb88 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:00.408985 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d5e4b0-3be0-4a29-a5d1-8fa65b60bb88 None None] PATH_INFO: `/` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:00.408985 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 216/439] 10.222.0.55 () {54 vars in 927 bytes} [Fri Sep 23 14:59:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 14:59:00.408985 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d5e4b0-3be0-4a29-a5d1-8fa65b60bb88 None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:00.420290 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 224/440] 10.222.0.55 () {58 vars in 1282 bytes} [Fri Sep 23 14:59:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 23 14:59:00.427711 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07b06a5-3b0d-4cab-9858-6861faacb2e1 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:00.428022 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07b06a5-3b0d-4cab-9858-6861faacb2e1 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:00.428306 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07b06a5-3b0d-4cab-9858-6861faacb2e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:00.453239 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-c07b06a5-3b0d-4cab-9858-6861faacb2e1 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 14:59:00.455395 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-c07b06a5-3b0d-4cab-9858-6861faacb2e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 14:59:00.531881 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 225/441] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Sep 23 14:59:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 14:59:00.533051 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07b06a5-3b0d-4cab-9858-6861faacb2e1 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:00.545315 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d5e4b0-3be0-4a29-a5d1-8fa65b60bb88 None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:00.561746 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:00.582966 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a47cf3b-d482-4a42-b33c-0466b0c15ed6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:00.583587 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a47cf3b-d482-4a42-b33c-0466b0c15ed6 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:00.583868 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a47cf3b-d482-4a42-b33c-0466b0c15ed6 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:00.584188 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a47cf3b-d482-4a42-b33c-0466b0c15ed6 None admin] PATH_INFO: `/v3/roles` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:00.584852 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a47cf3b-d482-4a42-b33c-0466b0c15ed6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:59:00.585191 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a47cf3b-d482-4a42-b33c-0466b0c15ed6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:00.586608 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a47cf3b-d482-4a42-b33c-0466b0c15ed6 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:00.601905 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 217/442] 10.222.0.55 () {56 vars in 1099 bytes} [Fri Sep 23 14:59:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 14:59:00.603136 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a47cf3b-d482-4a42-b33c-0466b0c15ed6 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:00.642200 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07b06a5-3b0d-4cab-9858-6861faacb2e1 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:00.670742 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:00.681654 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08d2654-3187-4713-a4bd-7cbb52fe7c5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:00.682326 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08d2654-3187-4713-a4bd-7cbb52fe7c5d None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:00.687289 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08d2654-3187-4713-a4bd-7cbb52fe7c5d None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:00.687289 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08d2654-3187-4713-a4bd-7cbb52fe7c5d None admin] PATH_INFO: `/v3/projects/0ffbe3ca86954853be08ecf627d6af00` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:00.687289 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08d2654-3187-4713-a4bd-7cbb52fe7c5d None admin] RBAC: Authorizing `identity:get_project(project_id=0ffbe3ca86954853be08ecf627d6af00)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:00.687289 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08d2654-3187-4713-a4bd-7cbb52fe7c5d None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:00.691969 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 226/443] 10.222.0.55 () {56 vars in 1186 bytes} [Fri Sep 23 14:59:00 2022] GET /identity/v3/projects/0ffbe3ca86954853be08ecf627d6af00 => generated 298 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:59:00.692817 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08d2654-3187-4713-a4bd-7cbb52fe7c5d None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:00.712757 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a47cf3b-d482-4a42-b33c-0466b0c15ed6 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:00.748277 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1184c639-278e-4976-97f8-fa98655909b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:00.748883 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1184c639-278e-4976-97f8-fa98655909b4 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:00.749108 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1184c639-278e-4976-97f8-fa98655909b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:00.749382 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1184c639-278e-4976-97f8-fa98655909b4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:00.797243 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:00.804145 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08d2654-3187-4713-a4bd-7cbb52fe7c5d None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:00.822284 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1184c639-278e-4976-97f8-fa98655909b4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:00.823752 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1184c639-278e-4976-97f8-fa98655909b4 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:00.840796 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f616573-d8ac-42ef-9178-0c4e2f18e7ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:00.841545 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f616573-d8ac-42ef-9178-0c4e2f18e7ee None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:00.842111 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f616573-d8ac-42ef-9178-0c4e2f18e7ee None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:00.842549 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f616573-d8ac-42ef-9178-0c4e2f18e7ee None admin] PATH_INFO: `/v3/groups/9980b9236dfe4cdabb8317a18b888217` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:00.843940 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f616573-d8ac-42ef-9178-0c4e2f18e7ee None admin] RBAC: Authorizing `identity:get_group(group_id=9980b9236dfe4cdabb8317a18b888217)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:00.845400 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f616573-d8ac-42ef-9178-0c4e2f18e7ee None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:00.847030 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 227/444] 10.222.0.55 () {56 vars in 1180 bytes} [Fri Sep 23 14:59:00 2022] GET /identity/v3/groups/9980b9236dfe4cdabb8317a18b888217 => generated 226 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:59:00.847819 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f616573-d8ac-42ef-9178-0c4e2f18e7ee None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:00.885168 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:00.925915 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 218/445] 10.222.0.55 () {58 vars in 1282 bytes} [Fri Sep 23 14:59:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 23 14:59:00.926755 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1184c639-278e-4976-97f8-fa98655909b4 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:00.949882 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f616573-d8ac-42ef-9178-0c4e2f18e7ee None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:00.983675 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac5778f-9739-4dc2-bc5e-3a2d7cc573af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:00.984328 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac5778f-9739-4dc2-bc5e-3a2d7cc573af None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:00.984608 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac5778f-9739-4dc2-bc5e-3a2d7cc573af None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:00.984878 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac5778f-9739-4dc2-bc5e-3a2d7cc573af None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:00.994839 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac5778f-9739-4dc2-bc5e-3a2d7cc573af None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 14:59:00.999603 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-3ac5778f-9739-4dc2-bc5e-3a2d7cc573af None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 23 14:59:00.999603 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 14:59:00.999603 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 14:59:00.999603 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 14:59:00.999603 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 14:59:00.999603 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 14:59:00.999603 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 14:59:00.999603 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 14:59:00.999603 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 14:59:00.999603 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 14:59:00.999603 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 14:59:00.999603 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 14:59:00.999603 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 14:59:00.999603 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 14:59:00.999603 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 14:59:00.999603 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 14:59:01.000551 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 14:59:01.000551 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 14:59:01.000551 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 14:59:01.000551 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 14:59:01.000551 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 14:59:01.000551 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 14:59:01.000551 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 14:59:01.000551 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 14:59:01.000551 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 14:59:01.000551 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 14:59:01.000551 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 14:59:01.000551 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Sep 23 14:59:01.000551 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 14:59:01.000551 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Sep 23 14:59:01.000551 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 14:59:01.000551 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 14:59:01.001364 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 14:59:01.001364 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Sep 23 14:59:01.001364 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 14:59:01.001364 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Sep 23 14:59:01.001364 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 14:59:01.001364 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 14:59:01.001364 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 14:59:01.001364 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 14:59:01.001364 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 14:59:01.001364 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 14:59:01.001364 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 23 14:59:01.001364 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Sep 23 14:59:01.001364 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 228/446] 10.222.0.55 () {56 vars in 1114 bytes} [Fri Sep 23 14:59:00 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 14:59:01.002027 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac5778f-9739-4dc2-bc5e-3a2d7cc573af None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:01.026086 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:01.028770 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1184c639-278e-4976-97f8-fa98655909b4 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:01.072224 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c220fbae-bca0-449a-a64f-1892ae4d781d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:01.072846 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c220fbae-bca0-449a-a64f-1892ae4d781d None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:01.073263 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c220fbae-bca0-449a-a64f-1892ae4d781d None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:01.073547 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c220fbae-bca0-449a-a64f-1892ae4d781d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:01.074282 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c220fbae-bca0-449a-a64f-1892ae4d781d None admin] RBAC: Adding query filter params (group.id=9980b9236dfe4cdabb8317a18b888217, role.id=af257be2dee54d6b942f58650196582c, scope.project.id=0ffbe3ca86954853be08ecf627d6af00) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:59:01.074593 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c220fbae-bca0-449a-a64f-1892ae4d781d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:01.076396 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c220fbae-bca0-449a-a64f-1892ae4d781d None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:01.086275 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 219/447] 10.222.0.55 () {56 vars in 1376 bytes} [Fri Sep 23 14:59:00 2022] GET /identity/v3/role_assignments?group.id=9980b9236dfe4cdabb8317a18b888217&scope.project.id=0ffbe3ca86954853be08ecf627d6af00&role.id=af257be2dee54d6b942f58650196582c => generated 611 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:59:01.087179 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c220fbae-bca0-449a-a64f-1892ae4d781d None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:01.113910 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac5778f-9739-4dc2-bc5e-3a2d7cc573af None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:01.119955 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:01.154344 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a59ad5-3707-4c05-b0b8-df482b74b46b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:01.154897 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a59ad5-3707-4c05-b0b8-df482b74b46b None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:01.155182 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a59ad5-3707-4c05-b0b8-df482b74b46b None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:01.155592 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a59ad5-3707-4c05-b0b8-df482b74b46b None admin] PATH_INFO: `/v3/projects/23b5c58d198d4456bfeec0a8daae1113` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:01.156841 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a59ad5-3707-4c05-b0b8-df482b74b46b None admin] RBAC: Authorizing `identity:get_project(project_id=23b5c58d198d4456bfeec0a8daae1113)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:01.158634 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a59ad5-3707-4c05-b0b8-df482b74b46b None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:01.160661 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 229/448] 10.222.0.55 () {56 vars in 1186 bytes} [Fri Sep 23 14:59:01 2022] GET /identity/v3/projects/23b5c58d198d4456bfeec0a8daae1113 => generated 344 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:59:01.168544 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a59ad5-3707-4c05-b0b8-df482b74b46b None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:01.213215 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c220fbae-bca0-449a-a64f-1892ae4d781d None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:01.252424 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9917c6a3-5ec2-4fde-9a81-2caf24fed80e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:01.253048 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9917c6a3-5ec2-4fde-9a81-2caf24fed80e None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:01.253330 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9917c6a3-5ec2-4fde-9a81-2caf24fed80e None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:01.253605 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9917c6a3-5ec2-4fde-9a81-2caf24fed80e None admin] PATH_INFO: `/v3/roles` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:01.254273 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9917c6a3-5ec2-4fde-9a81-2caf24fed80e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:59:01.254581 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9917c6a3-5ec2-4fde-9a81-2caf24fed80e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:01.255949 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9917c6a3-5ec2-4fde-9a81-2caf24fed80e None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:01.266419 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 220/449] 10.222.0.55 () {56 vars in 1111 bytes} [Fri Sep 23 14:59:01 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:59:01.272099 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9917c6a3-5ec2-4fde-9a81-2caf24fed80e None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:01.280996 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:01.294958 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a59ad5-3707-4c05-b0b8-df482b74b46b None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:01.341709 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d31d5b-970e-4eb6-b69a-74779a9009d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:01.342356 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d31d5b-970e-4eb6-b69a-74779a9009d5 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:01.342643 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d31d5b-970e-4eb6-b69a-74779a9009d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:01.342916 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d31d5b-970e-4eb6-b69a-74779a9009d5 None admin] PATH_INFO: `/v3/groups/0f32957856bc433188452103c17b1175` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:01.344583 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d31d5b-970e-4eb6-b69a-74779a9009d5 None admin] RBAC: Authorizing `identity:get_group(group_id=0f32957856bc433188452103c17b1175)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:01.345676 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d31d5b-970e-4eb6-b69a-74779a9009d5 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:01.347254 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 230/450] 10.222.0.55 () {56 vars in 1180 bytes} [Fri Sep 23 14:59:01 2022] GET /identity/v3/groups/0f32957856bc433188452103c17b1175 => generated 229 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 14:59:01.353548 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d31d5b-970e-4eb6-b69a-74779a9009d5 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:01.377487 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:01.390963 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9917c6a3-5ec2-4fde-9a81-2caf24fed80e None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:01.434131 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df755226-bdcc-4adb-bf85-0c2e0a36fe4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:01.434671 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df755226-bdcc-4adb-bf85-0c2e0a36fe4e None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:01.434954 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df755226-bdcc-4adb-bf85-0c2e0a36fe4e None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:01.435227 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df755226-bdcc-4adb-bf85-0c2e0a36fe4e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:01.482178 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d31d5b-970e-4eb6-b69a-74779a9009d5 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:01.530632 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df755226-bdcc-4adb-bf85-0c2e0a36fe4e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:01.532449 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df755226-bdcc-4adb-bf85-0c2e0a36fe4e None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:01.533014 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc565355-48e1-4722-9df2-6e6afad81f59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:01.533641 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc565355-48e1-4722-9df2-6e6afad81f59 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:01.533925 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc565355-48e1-4722-9df2-6e6afad81f59 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:01.534203 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc565355-48e1-4722-9df2-6e6afad81f59 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:01.534950 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc565355-48e1-4722-9df2-6e6afad81f59 None admin] RBAC: Adding query filter params (group.id=0f32957856bc433188452103c17b1175, role.id=8e2919e79df040e8926f21607818b58c, scope.project.id=23b5c58d198d4456bfeec0a8daae1113) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:59:01.535277 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc565355-48e1-4722-9df2-6e6afad81f59 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:01.536862 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc565355-48e1-4722-9df2-6e6afad81f59 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:01.543664 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 231/451] 10.222.0.55 () {56 vars in 1376 bytes} [Fri Sep 23 14:59:01 2022] GET /identity/v3/role_assignments?group.id=0f32957856bc433188452103c17b1175&scope.project.id=23b5c58d198d4456bfeec0a8daae1113&role.id=8e2919e79df040e8926f21607818b58c => generated 611 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 14:59:01.661823 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:01.679535 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 221/452] 10.222.0.55 () {58 vars in 1282 bytes} [Fri Sep 23 14:59:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 23 14:59:01.688169 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc565355-48e1-4722-9df2-6e6afad81f59 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:01.722911 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:01.828812 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc565355-48e1-4722-9df2-6e6afad81f59 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:01.871317 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cb4392-951e-426f-9d67-4bc1ee5dbf26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:01.871961 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cb4392-951e-426f-9d67-4bc1ee5dbf26 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:01.872277 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cb4392-951e-426f-9d67-4bc1ee5dbf26 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:01.872556 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cb4392-951e-426f-9d67-4bc1ee5dbf26 None admin] PATH_INFO: `/v3/projects/346756b911774d55af2a9ad3de2143b4` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:01.873808 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07cb4392-951e-426f-9d67-4bc1ee5dbf26 None admin] RBAC: Authorizing `identity:get_project(project_id=346756b911774d55af2a9ad3de2143b4)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:01.875231 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07cb4392-951e-426f-9d67-4bc1ee5dbf26 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:01.876920 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 232/453] 10.222.0.55 () {56 vars in 1186 bytes} [Fri Sep 23 14:59:01 2022] GET /identity/v3/projects/346756b911774d55af2a9ad3de2143b4 => generated 302 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:59:01.883104 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df755226-bdcc-4adb-bf85-0c2e0a36fe4e None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:01.990491 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df755226-bdcc-4adb-bf85-0c2e0a36fe4e None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:02.029007 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb77dca4-3fd7-48c9-a1d4-82a8ab1670a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:02.029947 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb77dca4-3fd7-48c9-a1d4-82a8ab1670a0 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:02.030262 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb77dca4-3fd7-48c9-a1d4-82a8ab1670a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:02.030586 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb77dca4-3fd7-48c9-a1d4-82a8ab1670a0 None admin] PATH_INFO: `/v3/groups/9980b9236dfe4cdabb8317a18b888217` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:02.032434 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb77dca4-3fd7-48c9-a1d4-82a8ab1670a0 None admin] RBAC: Authorizing `identity:get_group(group_id=9980b9236dfe4cdabb8317a18b888217)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:02.033966 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb77dca4-3fd7-48c9-a1d4-82a8ab1670a0 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:02.035526 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 222/454] 10.222.0.55 () {56 vars in 1180 bytes} [Fri Sep 23 14:59:01 2022] GET /identity/v3/groups/9980b9236dfe4cdabb8317a18b888217 => generated 226 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 14:59:02.043401 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cb4392-951e-426f-9d67-4bc1ee5dbf26 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:02.143316 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:02.153147 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cb4392-951e-426f-9d67-4bc1ee5dbf26 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:02.187193 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f38a1f-5736-4e93-8aa4-18aad5da7802 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:02.187785 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f38a1f-5736-4e93-8aa4-18aad5da7802 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:02.188098 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f38a1f-5736-4e93-8aa4-18aad5da7802 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:02.188378 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f38a1f-5736-4e93-8aa4-18aad5da7802 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:02.189251 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f38a1f-5736-4e93-8aa4-18aad5da7802 None admin] RBAC: Adding query filter params (group.id=9980b9236dfe4cdabb8317a18b888217, role.id=5c74fdc0e9cc44538cd8282006ed4443, scope.project.id=346756b911774d55af2a9ad3de2143b4) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:59:02.189564 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f38a1f-5736-4e93-8aa4-18aad5da7802 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:02.191027 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f38a1f-5736-4e93-8aa4-18aad5da7802 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:02.197620 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 233/455] 10.222.0.55 () {56 vars in 1376 bytes} [Fri Sep 23 14:59:02 2022] GET /identity/v3/role_assignments?group.id=9980b9236dfe4cdabb8317a18b888217&scope.project.id=346756b911774d55af2a9ad3de2143b4&role.id=5c74fdc0e9cc44538cd8282006ed4443 => generated 611 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:59:05.365074 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f7c9d9-8ff4-482d-bcbf-045b2829f9eb None None] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:05.366070 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f7c9d9-8ff4-482d-bcbf-045b2829f9eb None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:05.366588 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f7c9d9-8ff4-482d-bcbf-045b2829f9eb None None] PATH_INFO: `/` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:05.369217 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 223/456] 10.222.0.55 () {54 vars in 927 bytes} [Fri Sep 23 14:59:05 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 14:59:05.378145 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a02f47-2e67-450a-82ca-fedd6cf3d8d7 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:05.378145 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a02f47-2e67-450a-82ca-fedd6cf3d8d7 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:05.378145 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a02f47-2e67-450a-82ca-fedd6cf3d8d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:05.402562 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-d6a02f47-2e67-450a-82ca-fedd6cf3d8d7 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 14:59:05.404080 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-d6a02f47-2e67-450a-82ca-fedd6cf3d8d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 14:59:05.450834 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fe2fc7-80ef-4370-b186-b594d7864df1 None None] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:05.450834 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fe2fc7-80ef-4370-b186-b594d7864df1 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:05.450834 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fe2fc7-80ef-4370-b186-b594d7864df1 None None] PATH_INFO: `/` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:05.451517 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 224/457] 10.222.0.55 () {54 vars in 927 bytes} [Fri Sep 23 14:59:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 14:59:05.465687 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f511d55d-4bae-4aeb-ae91-508cff57905f None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:05.465989 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f511d55d-4bae-4aeb-ae91-508cff57905f None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:05.466233 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f511d55d-4bae-4aeb-ae91-508cff57905f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:05.491303 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-f511d55d-4bae-4aeb-ae91-508cff57905f None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 14:59:05.492948 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-f511d55d-4bae-4aeb-ae91-508cff57905f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 14:59:05.512065 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 234/458] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Sep 23 14:59:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 14:59:05.512065 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ef556f-28db-4e93-abe1-50966ae9c68b None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:05.512065 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ef556f-28db-4e93-abe1-50966ae9c68b None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:05.512065 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ef556f-28db-4e93-abe1-50966ae9c68b None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:05.512065 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 235/459] 10.222.0.55 () {54 vars in 927 bytes} [Fri Sep 23 14:59:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 14:59:05.517183 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d63d9e1-6ec1-4391-8e86-54834a4579ad None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:05.517468 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d63d9e1-6ec1-4391-8e86-54834a4579ad None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:05.517745 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d63d9e1-6ec1-4391-8e86-54834a4579ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:05.540535 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-9d63d9e1-6ec1-4391-8e86-54834a4579ad None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 14:59:05.541887 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-9d63d9e1-6ec1-4391-8e86-54834a4579ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 14:59:05.592351 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:05.624093 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 236/460] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Sep 23 14:59:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 23 14:59:05.631818 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 225/461] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Sep 23 14:59:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 23 14:59:05.854869 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d63d9e1-6ec1-4391-8e86-54834a4579ad None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:05.871817 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:05.965820 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d63d9e1-6ec1-4391-8e86-54834a4579ad None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:05.982571 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0d0d95-e666-44f1-a07e-ea1030d64b30 None None] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:05.983151 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0d0d95-e666-44f1-a07e-ea1030d64b30 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:05.983704 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0d0d95-e666-44f1-a07e-ea1030d64b30 None None] PATH_INFO: `/` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:05.985403 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 226/462] 10.222.0.55 () {54 vars in 927 bytes} [Fri Sep 23 14:59:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 23 14:59:05.997790 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffea863-3605-4c50-af73-264e74cf6eb0 None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:05.998414 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffea863-3605-4c50-af73-264e74cf6eb0 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:05.998909 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffea863-3605-4c50-af73-264e74cf6eb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:06.007999 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10e030a-e7f0-4523-92e0-1752050c12d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:06.007999 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10e030a-e7f0-4523-92e0-1752050c12d8 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:06.007999 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10e030a-e7f0-4523-92e0-1752050c12d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:06.007999 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10e030a-e7f0-4523-92e0-1752050c12d8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:06.014456 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10e030a-e7f0-4523-92e0-1752050c12d8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:06.015906 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10e030a-e7f0-4523-92e0-1752050c12d8 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:06.034810 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-d10e030a-e7f0-4523-92e0-1752050c12d8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 14:59:06.034810 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 14:59:06.034810 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 14:59:06.034810 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 14:59:06.034810 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 14:59:06.034810 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 14:59:06.034810 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 14:59:06.034810 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 14:59:06.034810 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 14:59:06.034810 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 14:59:06.034810 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 14:59:06.034810 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 14:59:06.034810 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 14:59:06.034810 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 14:59:06.034810 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 14:59:06.034810 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 14:59:06.035825 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 14:59:06.035825 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 14:59:06.035825 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 14:59:06.035825 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 14:59:06.035825 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 14:59:06.035825 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 14:59:06.035825 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 14:59:06.035825 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Sep 23 14:59:06.035825 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 14:59:06.035825 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Sep 23 14:59:06.035825 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 14:59:06.035825 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 14:59:06.035825 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 14:59:06.035825 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Sep 23 14:59:06.035825 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 14:59:06.035825 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Sep 23 14:59:06.035825 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 14:59:06.036724 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 14:59:06.036724 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 14:59:06.036724 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Sep 23 14:59:06.036724 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 237/463] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Sep 23 14:59:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 14:59:06.036939 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-0ffea863-3605-4c50-af73-264e74cf6eb0 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 14:59:06.037010 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10e030a-e7f0-4523-92e0-1752050c12d8 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:06.039692 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-0ffea863-3605-4c50-af73-264e74cf6eb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 14:59:06.116888 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:06.121177 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 227/464] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Sep 23 14:59:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 14:59:06.122193 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffea863-3605-4c50-af73-264e74cf6eb0 None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:06.165223 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10e030a-e7f0-4523-92e0-1752050c12d8 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:06.205482 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da50ec2-5ada-481b-bab3-33e1adc2313f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:06.206204 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da50ec2-5ada-481b-bab3-33e1adc2313f None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:06.206489 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da50ec2-5ada-481b-bab3-33e1adc2313f None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:06.207019 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da50ec2-5ada-481b-bab3-33e1adc2313f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:06.221223 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da50ec2-5ada-481b-bab3-33e1adc2313f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:06.222624 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da50ec2-5ada-481b-bab3-33e1adc2313f None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:06.244505 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-1da50ec2-5ada-481b-bab3-33e1adc2313f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 14:59:06.244505 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 14:59:06.244505 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 14:59:06.244505 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 14:59:06.244505 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 14:59:06.244505 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 14:59:06.244505 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 14:59:06.244505 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 14:59:06.244505 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 14:59:06.244505 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 14:59:06.244505 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 14:59:06.244505 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 14:59:06.244505 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 14:59:06.244505 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 14:59:06.244505 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 14:59:06.244505 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 14:59:06.245637 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 14:59:06.245637 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 14:59:06.245637 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 14:59:06.245637 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 14:59:06.245637 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 14:59:06.245637 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 14:59:06.245637 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 14:59:06.245637 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Sep 23 14:59:06.245637 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 14:59:06.245637 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Sep 23 14:59:06.245637 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 14:59:06.245637 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 14:59:06.245637 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 14:59:06.245637 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Sep 23 14:59:06.245637 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 14:59:06.245637 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Sep 23 14:59:06.245637 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 14:59:06.246470 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 14:59:06.246470 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 14:59:06.246470 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Sep 23 14:59:06.253215 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 238/465] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Sep 23 14:59:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 14:59:06.253215 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da50ec2-5ada-481b-bab3-33e1adc2313f None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:06.266751 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffea863-3605-4c50-af73-264e74cf6eb0 None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:06.290549 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:06.313880 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a916d7c-34a2-46b8-acb0-bd790935c05c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:06.314506 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a916d7c-34a2-46b8-acb0-bd790935c05c None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:06.314787 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a916d7c-34a2-46b8-acb0-bd790935c05c None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:06.315063 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a916d7c-34a2-46b8-acb0-bd790935c05c None admin] PATH_INFO: `/v3/domains` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:06.315754 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a916d7c-34a2-46b8-acb0-bd790935c05c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:59:06.316338 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a916d7c-34a2-46b8-acb0-bd790935c05c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:06.318168 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a916d7c-34a2-46b8-acb0-bd790935c05c None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:06.335243 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 228/466] 10.222.0.55 () {56 vars in 1109 bytes} [Fri Sep 23 14:59:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:59:06.336272 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a916d7c-34a2-46b8-acb0-bd790935c05c None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:06.380827 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da50ec2-5ada-481b-bab3-33e1adc2313f None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:06.406054 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:06.426384 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad17ef3c-3f40-4d54-a528-b1e556984cd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:06.427022 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad17ef3c-3f40-4d54-a528-b1e556984cd0 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:06.427301 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad17ef3c-3f40-4d54-a528-b1e556984cd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:06.427673 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad17ef3c-3f40-4d54-a528-b1e556984cd0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:06.438606 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad17ef3c-3f40-4d54-a528-b1e556984cd0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:06.440086 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad17ef3c-3f40-4d54-a528-b1e556984cd0 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:06.455491 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-ad17ef3c-3f40-4d54-a528-b1e556984cd0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 14:59:06.455491 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 14:59:06.455491 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 14:59:06.455491 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 14:59:06.455491 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 14:59:06.455491 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 14:59:06.455491 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 14:59:06.455491 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 14:59:06.455491 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 14:59:06.455491 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 14:59:06.455491 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 14:59:06.455491 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 14:59:06.455491 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 14:59:06.455491 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 14:59:06.455491 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 14:59:06.455491 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 14:59:06.456464 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 14:59:06.456464 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 14:59:06.456464 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 14:59:06.456464 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 14:59:06.456464 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 14:59:06.456464 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 14:59:06.456464 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 14:59:06.456464 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Sep 23 14:59:06.456464 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 14:59:06.456464 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Sep 23 14:59:06.456464 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 14:59:06.456464 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 14:59:06.456464 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 14:59:06.456464 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Sep 23 14:59:06.456464 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 14:59:06.456464 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Sep 23 14:59:06.456464 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 14:59:06.457336 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 14:59:06.457336 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 14:59:06.457336 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Sep 23 14:59:06.457336 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 239/467] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Sep 23 14:59:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 14:59:06.457336 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad17ef3c-3f40-4d54-a528-b1e556984cd0 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:06.468023 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a916d7c-34a2-46b8-acb0-bd790935c05c None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:06.503970 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fd9d5c-e2ec-40ca-95b1-ccb0047d4e68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:06.504775 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fd9d5c-e2ec-40ca-95b1-ccb0047d4e68 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:06.504955 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fd9d5c-e2ec-40ca-95b1-ccb0047d4e68 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:06.505230 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fd9d5c-e2ec-40ca-95b1-ccb0047d4e68 None admin] PATH_INFO: `/v3/domains` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:06.505885 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fd9d5c-e2ec-40ca-95b1-ccb0047d4e68 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:59:06.506189 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fd9d5c-e2ec-40ca-95b1-ccb0047d4e68 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:06.510332 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fd9d5c-e2ec-40ca-95b1-ccb0047d4e68 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:06.526133 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 229/468] 10.222.0.55 () {56 vars in 1109 bytes} [Fri Sep 23 14:59:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:59:06.526133 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fd9d5c-e2ec-40ca-95b1-ccb0047d4e68 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:06.562170 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:06.580871 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad17ef3c-3f40-4d54-a528-b1e556984cd0 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:06.623010 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0eafb8-bf0b-4961-bdae-063d309bb251 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:06.623740 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0eafb8-bf0b-4961-bdae-063d309bb251 None admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:06.624062 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0eafb8-bf0b-4961-bdae-063d309bb251 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:06.624345 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0eafb8-bf0b-4961-bdae-063d309bb251 None admin] PATH_INFO: `/v3/users` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:06.625012 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0eafb8-bf0b-4961-bdae-063d309bb251 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:06.626492 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0eafb8-bf0b-4961-bdae-063d309bb251 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:06.641930 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fd9d5c-e2ec-40ca-95b1-ccb0047d4e68 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:06.677998 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59e2915-a36b-4798-a58e-df0ee97dd2eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:06.678635 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59e2915-a36b-4798-a58e-df0ee97dd2eb None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:06.678917 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59e2915-a36b-4798-a58e-df0ee97dd2eb None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:06.679190 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59e2915-a36b-4798-a58e-df0ee97dd2eb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:06.690085 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59e2915-a36b-4798-a58e-df0ee97dd2eb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:06.691549 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59e2915-a36b-4798-a58e-df0ee97dd2eb None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:06.707226 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application [None req-f59e2915-a36b-4798-a58e-df0ee97dd2eb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 14:59:06.707226 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 14:59:06.707226 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 14:59:06.707226 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 14:59:06.707226 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 14:59:06.707226 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 14:59:06.707226 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 14:59:06.707226 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 14:59:06.707226 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 14:59:06.707226 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 14:59:06.707226 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 14:59:06.707226 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 14:59:06.707226 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 14:59:06.707226 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 14:59:06.707226 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 14:59:06.707226 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 14:59:06.708356 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 14:59:06.708356 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 14:59:06.708356 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 14:59:06.708356 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 14:59:06.708356 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 14:59:06.708356 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 14:59:06.708356 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 14:59:06.708356 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application with Lock( Sep 23 14:59:06.708356 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 14:59:06.708356 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._enter() Sep 23 14:59:06.708356 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 14:59:06.708356 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 14:59:06.708356 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 14:59:06.708356 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.creator() Sep 23 14:59:06.708356 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 14:59:06.708356 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application created_value = creator( Sep 23 14:59:06.708356 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 14:59:06.709224 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 14:59:06.709224 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 14:59:06.709224 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Sep 23 14:59:06.709224 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 230/469] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Sep 23 14:59:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 14:59:06.709224 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59e2915-a36b-4798-a58e-df0ee97dd2eb None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:06.775236 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 240/470] 10.222.0.55 () {60 vars in 1132 bytes} [Fri Sep 23 14:59:06 2022] POST /identity/v3/users => generated 246 bytes in 318 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 14:59:06.776505 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0eafb8-bf0b-4961-bdae-063d309bb251 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:06.812855 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:06.818414 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:06.836042 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59e2915-a36b-4798-a58e-df0ee97dd2eb None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:06.874472 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b7888d-8b17-4a7d-8549-a762275bb73a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:06.875110 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b7888d-8b17-4a7d-8549-a762275bb73a None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:06.875686 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b7888d-8b17-4a7d-8549-a762275bb73a None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:06.875901 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b7888d-8b17-4a7d-8549-a762275bb73a None admin] PATH_INFO: `/v3/domains` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:06.876675 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b7888d-8b17-4a7d-8549-a762275bb73a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:59:06.877552 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b7888d-8b17-4a7d-8549-a762275bb73a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:06.879014 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b7888d-8b17-4a7d-8549-a762275bb73a None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:06.897401 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 231/471] 10.222.0.55 () {56 vars in 1109 bytes} [Fri Sep 23 14:59:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:59:06.897401 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b7888d-8b17-4a7d-8549-a762275bb73a None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:06.911956 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0eafb8-bf0b-4961-bdae-063d309bb251 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:06.952497 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b694fe3-f7b3-4bac-a34a-4c15670a5037 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:06.953185 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b694fe3-f7b3-4bac-a34a-4c15670a5037 None admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:06.953470 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b694fe3-f7b3-4bac-a34a-4c15670a5037 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:06.953749 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b694fe3-f7b3-4bac-a34a-4c15670a5037 None admin] PATH_INFO: `/v3/users` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:06.954452 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b694fe3-f7b3-4bac-a34a-4c15670a5037 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:06.955864 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b694fe3-f7b3-4bac-a34a-4c15670a5037 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:07.007572 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b7888d-8b17-4a7d-8549-a762275bb73a None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:07.044464 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8faa077-96de-43fb-96d7-5bbbca0c0f36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:07.045005 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8faa077-96de-43fb-96d7-5bbbca0c0f36 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:07.045285 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8faa077-96de-43fb-96d7-5bbbca0c0f36 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:07.045562 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8faa077-96de-43fb-96d7-5bbbca0c0f36 None admin] PATH_INFO: `/v3/domains` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:07.046205 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8faa077-96de-43fb-96d7-5bbbca0c0f36 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:59:07.046509 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8faa077-96de-43fb-96d7-5bbbca0c0f36 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:07.048065 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8faa077-96de-43fb-96d7-5bbbca0c0f36 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:07.048569 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 241/472] 10.222.0.55 () {60 vars in 1132 bytes} [Fri Sep 23 14:59:06 2022] POST /identity/v3/users => generated 248 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 14:59:07.052077 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dbf055-18bd-4407-9d62-f48b70e2effe None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:07.052077 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dbf055-18bd-4407-9d62-f48b70e2effe None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:07.052077 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dbf055-18bd-4407-9d62-f48b70e2effe None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:07.052328 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 242/473] 10.222.0.55 () {54 vars in 927 bytes} [Fri Sep 23 14:59:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 14:59:07.059113 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67dbf055-18bd-4407-9d62-f48b70e2effe None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:07.064487 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 232/474] 10.222.0.55 () {56 vars in 1109 bytes} [Fri Sep 23 14:59:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:59:07.064487 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11c80bc-f994-4326-92f7-1606f7f53322 None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:07.064738 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11c80bc-f994-4326-92f7-1606f7f53322 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:07.065016 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11c80bc-f994-4326-92f7-1606f7f53322 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:07.101066 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-d11c80bc-f994-4326-92f7-1606f7f53322 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 14:59:07.103365 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-d11c80bc-f994-4326-92f7-1606f7f53322 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 14:59:07.107269 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:07.191785 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:07.206930 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 233/475] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Sep 23 14:59:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 14:59:07.208241 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11c80bc-f994-4326-92f7-1606f7f53322 None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:07.215617 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67dbf055-18bd-4407-9d62-f48b70e2effe None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:07.255479 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77918eb-b23f-481d-8884-9588f124bc33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:07.256443 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77918eb-b23f-481d-8884-9588f124bc33 None admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:07.256957 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77918eb-b23f-481d-8884-9588f124bc33 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:07.257405 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77918eb-b23f-481d-8884-9588f124bc33 None admin] PATH_INFO: `/v3/users` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:07.258230 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77918eb-b23f-481d-8884-9588f124bc33 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:07.260842 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77918eb-b23f-481d-8884-9588f124bc33 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:07.327394 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11c80bc-f994-4326-92f7-1606f7f53322 None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:07.364059 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1554ccba-fdbd-46d3-b4a0-546dcaa42577 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:07.364736 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1554ccba-fdbd-46d3-b4a0-546dcaa42577 None admin] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:07.365019 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1554ccba-fdbd-46d3-b4a0-546dcaa42577 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:07.365292 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1554ccba-fdbd-46d3-b4a0-546dcaa42577 None admin] PATH_INFO: `/v3/users` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:07.365940 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1554ccba-fdbd-46d3-b4a0-546dcaa42577 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:07.367477 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1554ccba-fdbd-46d3-b4a0-546dcaa42577 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:07.380756 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:07.403747 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 243/476] 10.222.0.55 () {60 vars in 1132 bytes} [Fri Sep 23 14:59:07 2022] POST /identity/v3/users => generated 248 bytes in 351 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 14:59:07.467113 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 234/477] 10.222.0.55 () {60 vars in 1132 bytes} [Fri Sep 23 14:59:07 2022] POST /identity/v3/users => generated 249 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 14:59:07.521362 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77918eb-b23f-481d-8884-9588f124bc33 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:07.673699 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77918eb-b23f-481d-8884-9588f124bc33 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:07.716712 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771531e2-c889-4692-8645-b794cec00a54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:07.717338 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771531e2-c889-4692-8645-b794cec00a54 None admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:07.717620 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771531e2-c889-4692-8645-b794cec00a54 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:07.717889 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771531e2-c889-4692-8645-b794cec00a54 None admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:07.718639 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771531e2-c889-4692-8645-b794cec00a54 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:07.720047 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771531e2-c889-4692-8645-b794cec00a54 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:07.760741 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.sql.core [None req-771531e2-c889-4692-8645-b794cec00a54 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 23 14:59:07.760741 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 23 14:59:07.760741 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [parameters: {'id': '2d5bbf1741014e49a5804a9b1807fe74', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 23 14:59:07.760741 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81994) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 23 14:59:07.778191 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-771531e2-c889-4692-8645-b794cec00a54 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 23 14:59:07.778191 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 14:59:07.778191 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 23 14:59:07.778191 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 23 14:59:07.778191 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 23 14:59:07.778191 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 23 14:59:07.778191 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 23 14:59:07.778191 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application result = self._query(query) Sep 23 14:59:07.778191 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 23 14:59:07.778191 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application conn.query(q) Sep 23 14:59:07.778191 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 23 14:59:07.778191 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 23 14:59:07.778191 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 23 14:59:07.778191 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application result.read() Sep 23 14:59:07.778191 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 23 14:59:07.779894 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 23 14:59:07.779894 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 23 14:59:07.779894 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 23 14:59:07.779894 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 23 14:59:07.779894 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 23 14:59:07.779894 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 23 14:59:07.779894 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 23 14:59:07.779894 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 23 14:59:07.779894 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Sep 23 14:59:07.779894 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 23 14:59:07.779894 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Sep 23 14:59:07.779894 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 14:59:07.779894 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 23 14:59:07.779894 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 23 14:59:07.779894 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Sep 23 14:59:07.779894 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return ref.to_dict() Sep 23 14:59:07.779894 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 23 14:59:07.779894 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application next(self.gen) Sep 23 14:59:07.782365 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 23 14:59:07.782365 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application yield resource Sep 23 14:59:07.782365 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 23 14:59:07.782365 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application next(self.gen) Sep 23 14:59:07.782365 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 23 14:59:07.782365 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application self.session.rollback() Sep 23 14:59:07.782365 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 23 14:59:07.782365 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application self.force_reraise() Sep 23 14:59:07.782365 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 23 14:59:07.782365 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise self.value Sep 23 14:59:07.782365 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 23 14:59:07.782365 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 23 14:59:07.782365 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 23 14:59:07.782365 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application session.commit() Sep 23 14:59:07.782365 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 23 14:59:07.782365 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 23 14:59:07.782365 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 23 14:59:07.783226 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application self._prepare_impl() Sep 23 14:59:07.783226 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 23 14:59:07.783226 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application self.session.flush() Sep 23 14:59:07.783226 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 23 14:59:07.783226 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application self._flush(objects) Sep 23 14:59:07.783226 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 23 14:59:07.783226 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 23 14:59:07.783226 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 23 14:59:07.783226 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application compat.raise_( Sep 23 14:59:07.783226 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 23 14:59:07.783226 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception Sep 23 14:59:07.783226 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 23 14:59:07.783226 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application flush_context.execute() Sep 23 14:59:07.783226 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 23 14:59:07.783226 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rec.execute(self) Sep 23 14:59:07.783226 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 23 14:59:07.783226 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 23 14:59:07.784746 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 23 14:59:07.784746 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 23 14:59:07.784746 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 23 14:59:07.784746 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 23 14:59:07.784746 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 23 14:59:07.784746 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 23 14:59:07.784746 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 23 14:59:07.784746 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 23 14:59:07.784746 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 23 14:59:07.784746 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 23 14:59:07.784746 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 23 14:59:07.784746 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 23 14:59:07.784746 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 23 14:59:07.784746 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 23 14:59:07.784746 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 23 14:59:07.784746 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception Sep 23 14:59:07.788453 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 23 14:59:07.788453 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 23 14:59:07.788453 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 23 14:59:07.788453 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 23 14:59:07.788453 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 23 14:59:07.788453 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application result = self._query(query) Sep 23 14:59:07.788453 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 23 14:59:07.788453 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application conn.query(q) Sep 23 14:59:07.788453 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 23 14:59:07.788453 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 23 14:59:07.788453 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 23 14:59:07.788453 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application result.read() Sep 23 14:59:07.788453 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 23 14:59:07.788453 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 23 14:59:07.788453 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 23 14:59:07.788453 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 23 14:59:07.791239 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 23 14:59:07.791239 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 23 14:59:07.791239 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 23 14:59:07.791239 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 23 14:59:07.791239 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 23 14:59:07.791239 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 23 14:59:07.791239 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [parameters: {'id': '2d5bbf1741014e49a5804a9b1807fe74', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 23 14:59:07.791239 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 23 14:59:07.791239 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Sep 23 14:59:07.791239 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 23 14:59:07.791239 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Sep 23 14:59:07.791239 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 14:59:07.791239 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 14:59:07.791239 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 14:59:07.791239 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 14:59:07.791239 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 14:59:07.793333 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 14:59:07.793333 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 14:59:07.793333 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 14:59:07.793333 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 14:59:07.793333 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 14:59:07.793333 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 14:59:07.793333 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Sep 23 14:59:07.793333 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Sep 23 14:59:07.793333 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 14:59:07.793333 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 14:59:07.793333 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Sep 23 14:59:07.793333 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Sep 23 14:59:07.793333 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 23 14:59:07.793333 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 23 14:59:07.793333 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 23 14:59:07.793333 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Sep 23 14:59:07.795352 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 244/478] 10.222.0.55 () {60 vars in 1131 bytes} [Fri Sep 23 14:59:07 2022] POST /identity/v3/roles => generated 144 bytes in 259 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 23 14:59:07.795489 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1554ccba-fdbd-46d3-b4a0-546dcaa42577 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:07.912271 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1554ccba-fdbd-46d3-b4a0-546dcaa42577 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:07.953456 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f062486-2574-47ad-aaca-07440a152260 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:07.954034 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f062486-2574-47ad-aaca-07440a152260 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:07.954317 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f062486-2574-47ad-aaca-07440a152260 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:07.954597 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f062486-2574-47ad-aaca-07440a152260 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:07.966491 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f062486-2574-47ad-aaca-07440a152260 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 14:59:07.971928 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application [None req-7f062486-2574-47ad-aaca-07440a152260 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 23 14:59:07.971928 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 14:59:07.971928 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 14:59:07.971928 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 14:59:07.971928 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 14:59:07.971928 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 14:59:07.971928 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 14:59:07.971928 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 14:59:07.971928 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 14:59:07.971928 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 14:59:07.971928 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 14:59:07.971928 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 14:59:07.971928 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 14:59:07.971928 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 14:59:07.971928 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 14:59:07.971928 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 14:59:07.972866 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 14:59:07.972866 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 14:59:07.972866 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 14:59:07.972866 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 14:59:07.972866 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 14:59:07.972866 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 14:59:07.972866 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 14:59:07.972866 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 14:59:07.972866 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 14:59:07.972866 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 14:59:07.972866 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 14:59:07.972866 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application with Lock( Sep 23 14:59:07.972866 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 14:59:07.972866 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._enter() Sep 23 14:59:07.972866 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 14:59:07.972866 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 14:59:07.973654 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 14:59:07.973654 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.creator() Sep 23 14:59:07.973654 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 14:59:07.973654 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application created_value = creator( Sep 23 14:59:07.973654 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 14:59:07.973654 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 14:59:07.973654 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 14:59:07.973654 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 14:59:07.973654 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 14:59:07.973654 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 14:59:07.973654 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 23 14:59:07.973654 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Sep 23 14:59:07.973654 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 235/479] 10.222.0.55 () {56 vars in 1114 bytes} [Fri Sep 23 14:59:07 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 23 14:59:07.980323 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771531e2-c889-4692-8645-b794cec00a54 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:07.983307 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:08.109350 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771531e2-c889-4692-8645-b794cec00a54 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:08.150097 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c537c70b-bce6-4786-a56d-7300723dff83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:08.151775 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c537c70b-bce6-4786-a56d-7300723dff83 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:08.152321 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c537c70b-bce6-4786-a56d-7300723dff83 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:08.152814 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c537c70b-bce6-4786-a56d-7300723dff83 None admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:08.153924 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c537c70b-bce6-4786-a56d-7300723dff83 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:59:08.154456 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c537c70b-bce6-4786-a56d-7300723dff83 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:08.156085 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c537c70b-bce6-4786-a56d-7300723dff83 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:08.167725 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 245/480] 10.222.0.55 () {56 vars in 1111 bytes} [Fri Sep 23 14:59:07 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:59:09.308279 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a067609-0ddf-432e-99d2-765008734ad4 None None] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:09.308788 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a067609-0ddf-432e-99d2-765008734ad4 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:09.308788 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a067609-0ddf-432e-99d2-765008734ad4 None None] PATH_INFO: `/` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:09.310405 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 236/481] 10.222.0.55 () {54 vars in 927 bytes} [Fri Sep 23 14:59:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 14:59:09.315925 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84e708c-1698-42f0-826a-fb36ab2b2417 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:09.316337 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84e708c-1698-42f0-826a-fb36ab2b2417 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:09.316523 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84e708c-1698-42f0-826a-fb36ab2b2417 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:09.335212 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-f84e708c-1698-42f0-826a-fb36ab2b2417 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 14:59:09.336362 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-f84e708c-1698-42f0-826a-fb36ab2b2417 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 14:59:09.409834 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:09.422478 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 246/482] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Sep 23 14:59:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 14:59:09.800578 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a067609-0ddf-432e-99d2-765008734ad4 None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:09.863883 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:09.874796 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28923d4b-f72e-40de-8954-ed43e25610ac None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:09.875089 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28923d4b-f72e-40de-8954-ed43e25610ac None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:09.875365 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28923d4b-f72e-40de-8954-ed43e25610ac None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:09.877816 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 247/483] 10.222.0.55 () {54 vars in 927 bytes} [Fri Sep 23 14:59:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 14:59:09.881718 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e91fdff-73c6-4f20-b93b-bf73c56f46a8 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:09.882004 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e91fdff-73c6-4f20-b93b-bf73c56f46a8 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:09.882277 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e91fdff-73c6-4f20-b93b-bf73c56f46a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:09.912149 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a067609-0ddf-432e-99d2-765008734ad4 None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:09.927302 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-9e91fdff-73c6-4f20-b93b-bf73c56f46a8 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 14:59:09.928536 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-9e91fdff-73c6-4f20-b93b-bf73c56f46a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 14:59:09.948187 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9996d08-eca1-41e5-b700-31e931bdcbdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:09.948822 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9996d08-eca1-41e5-b700-31e931bdcbdf None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:09.949108 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9996d08-eca1-41e5-b700-31e931bdcbdf None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:09.949388 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9996d08-eca1-41e5-b700-31e931bdcbdf None admin] PATH_INFO: `/v3/roles/service` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:09.960502 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9996d08-eca1-41e5-b700-31e931bdcbdf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 14:59:09.966151 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application [None req-e9996d08-eca1-41e5-b700-31e931bdcbdf None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 23 14:59:09.966151 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 14:59:09.966151 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 14:59:09.966151 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 14:59:09.966151 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 14:59:09.966151 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 14:59:09.966151 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 14:59:09.966151 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 14:59:09.966151 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 14:59:09.966151 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 14:59:09.966151 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 14:59:09.966151 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 14:59:09.966151 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 14:59:09.966151 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 14:59:09.966151 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 14:59:09.966151 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 14:59:09.967073 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 14:59:09.967073 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 14:59:09.967073 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 14:59:09.967073 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 14:59:09.967073 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 14:59:09.967073 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 14:59:09.967073 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 14:59:09.967073 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 14:59:09.967073 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 14:59:09.967073 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 14:59:09.967073 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 14:59:09.967073 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application with Lock( Sep 23 14:59:09.967073 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 14:59:09.967073 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._enter() Sep 23 14:59:09.967073 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 14:59:09.967073 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 14:59:09.967884 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 14:59:09.967884 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.creator() Sep 23 14:59:09.967884 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 14:59:09.967884 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application created_value = creator( Sep 23 14:59:09.967884 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 14:59:09.967884 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 14:59:09.967884 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 14:59:09.967884 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 14:59:09.967884 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 14:59:09.967884 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 14:59:09.967884 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 23 14:59:09.967884 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Sep 23 14:59:09.967884 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 237/484] 10.222.0.55 () {56 vars in 1102 bytes} [Fri Sep 23 14:59:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 14:59:09.973614 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9996d08-eca1-41e5-b700-31e931bdcbdf None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:10.064274 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 248/485] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Sep 23 14:59:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 14:59:10.078028 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9996d08-eca1-41e5-b700-31e931bdcbdf None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:10.096406 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:10.114919 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0b100c-7071-45a2-83d3-c444af6d4f79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:10.115554 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0b100c-7071-45a2-83d3-c444af6d4f79 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:10.117599 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0b100c-7071-45a2-83d3-c444af6d4f79 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:10.117599 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0b100c-7071-45a2-83d3-c444af6d4f79 None admin] PATH_INFO: `/v3/roles` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:10.117599 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0b100c-7071-45a2-83d3-c444af6d4f79 None admin] RBAC: Adding query filter params (name=service) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:59:10.117599 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0b100c-7071-45a2-83d3-c444af6d4f79 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:10.119006 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0b100c-7071-45a2-83d3-c444af6d4f79 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:10.129890 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 238/486] 10.222.0.55 () {56 vars in 1103 bytes} [Fri Sep 23 14:59:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:59:10.136462 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e91fdff-73c6-4f20-b93b-bf73c56f46a8 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:10.270833 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e91fdff-73c6-4f20-b93b-bf73c56f46a8 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:10.303317 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:10.320862 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246eaac9-3719-4668-b783-4167f21b258d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:10.320862 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246eaac9-3719-4668-b783-4167f21b258d None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:10.320862 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246eaac9-3719-4668-b783-4167f21b258d None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:10.320862 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246eaac9-3719-4668-b783-4167f21b258d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:10.330366 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246eaac9-3719-4668-b783-4167f21b258d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:10.331834 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246eaac9-3719-4668-b783-4167f21b258d None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:10.348636 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-246eaac9-3719-4668-b783-4167f21b258d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 14:59:10.348636 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 14:59:10.348636 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 14:59:10.348636 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 14:59:10.348636 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 14:59:10.348636 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 14:59:10.348636 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 14:59:10.348636 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 14:59:10.348636 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 14:59:10.348636 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 14:59:10.348636 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 14:59:10.348636 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 14:59:10.348636 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 14:59:10.348636 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 14:59:10.348636 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 14:59:10.348636 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 14:59:10.349903 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 14:59:10.349903 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 14:59:10.349903 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 14:59:10.349903 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 14:59:10.349903 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 14:59:10.349903 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 14:59:10.349903 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 14:59:10.349903 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Sep 23 14:59:10.349903 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 14:59:10.349903 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Sep 23 14:59:10.349903 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 14:59:10.349903 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 14:59:10.349903 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 14:59:10.349903 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Sep 23 14:59:10.349903 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 14:59:10.349903 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Sep 23 14:59:10.349903 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 14:59:10.350984 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 14:59:10.350984 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 14:59:10.350984 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Sep 23 14:59:10.353361 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 249/487] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Sep 23 14:59:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 14:59:10.358895 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0b100c-7071-45a2-83d3-c444af6d4f79 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:10.371275 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210a164b-465f-4369-9b90-5f2649365aac None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:10.371583 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210a164b-465f-4369-9b90-5f2649365aac None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:10.371816 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210a164b-465f-4369-9b90-5f2649365aac None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:10.373167 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 250/488] 10.222.0.55 () {54 vars in 927 bytes} [Fri Sep 23 14:59:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 14:59:10.379792 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6502762d-9a08-4661-bfaf-bf35824a5ae5 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:10.380093 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6502762d-9a08-4661-bfaf-bf35824a5ae5 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:10.380371 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6502762d-9a08-4661-bfaf-bf35824a5ae5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:10.414193 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-6502762d-9a08-4661-bfaf-bf35824a5ae5 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 14:59:10.415709 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-6502762d-9a08-4661-bfaf-bf35824a5ae5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 14:59:10.486157 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0b100c-7071-45a2-83d3-c444af6d4f79 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:10.497818 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 251/489] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Sep 23 14:59:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 14:59:10.499503 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6502762d-9a08-4661-bfaf-bf35824a5ae5 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:10.523720 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c12084-dc03-4838-b0b6-753e826315d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:10.524530 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c12084-dc03-4838-b0b6-753e826315d6 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:10.525202 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c12084-dc03-4838-b0b6-753e826315d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:10.525202 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c12084-dc03-4838-b0b6-753e826315d6 None admin] PATH_INFO: `/v3/domains` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:10.525941 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c12084-dc03-4838-b0b6-753e826315d6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:59:10.526338 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c12084-dc03-4838-b0b6-753e826315d6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:10.527770 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c12084-dc03-4838-b0b6-753e826315d6 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:10.540962 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 239/490] 10.222.0.55 () {56 vars in 1109 bytes} [Fri Sep 23 14:59:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:59:10.547178 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c12084-dc03-4838-b0b6-753e826315d6 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:10.626735 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6502762d-9a08-4661-bfaf-bf35824a5ae5 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:10.635041 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:10.638080 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:10.667522 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca9f5ca-1510-4edf-a1d5-7bf939f8d8cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:10.668365 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca9f5ca-1510-4edf-a1d5-7bf939f8d8cb None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:10.668743 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca9f5ca-1510-4edf-a1d5-7bf939f8d8cb None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:10.669115 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca9f5ca-1510-4edf-a1d5-7bf939f8d8cb None admin] PATH_INFO: `/v3/roles/service` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:10.684337 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca9f5ca-1510-4edf-a1d5-7bf939f8d8cb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 14:59:10.689725 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-2ca9f5ca-1510-4edf-a1d5-7bf939f8d8cb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 23 14:59:10.689725 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 14:59:10.689725 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 14:59:10.689725 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 14:59:10.689725 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 14:59:10.689725 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 14:59:10.689725 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 14:59:10.689725 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 14:59:10.689725 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 14:59:10.689725 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 14:59:10.689725 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 14:59:10.689725 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 14:59:10.689725 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 14:59:10.689725 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 14:59:10.689725 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 14:59:10.689725 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 14:59:10.690680 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 14:59:10.690680 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 14:59:10.690680 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 14:59:10.690680 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 14:59:10.690680 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 14:59:10.690680 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 14:59:10.690680 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 14:59:10.690680 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 14:59:10.690680 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 14:59:10.690680 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 14:59:10.690680 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 14:59:10.690680 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Sep 23 14:59:10.690680 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 14:59:10.690680 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Sep 23 14:59:10.690680 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 14:59:10.690680 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 14:59:10.691478 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 14:59:10.691478 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Sep 23 14:59:10.691478 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 14:59:10.691478 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Sep 23 14:59:10.691478 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 14:59:10.691478 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 14:59:10.691478 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 14:59:10.691478 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 14:59:10.691478 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 14:59:10.691478 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 14:59:10.691478 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 23 14:59:10.691478 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Sep 23 14:59:10.691478 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 252/491] 10.222.0.55 () {56 vars in 1102 bytes} [Fri Sep 23 14:59:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 14:59:10.699148 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca9f5ca-1510-4edf-a1d5-7bf939f8d8cb None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:10.804685 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c12084-dc03-4838-b0b6-753e826315d6 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:10.842579 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d023bccd-c090-4502-bf80-8c70b83a386b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:10.843428 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d023bccd-c090-4502-bf80-8c70b83a386b None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:10.843625 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d023bccd-c090-4502-bf80-8c70b83a386b None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:10.843902 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d023bccd-c090-4502-bf80-8c70b83a386b None admin] PATH_INFO: `/v3/users/nova` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:10.852064 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d023bccd-c090-4502-bf80-8c70b83a386b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:10.854208 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d023bccd-c090-4502-bf80-8c70b83a386b None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:10.860129 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca9f5ca-1510-4edf-a1d5-7bf939f8d8cb None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:10.866710 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application [None req-d023bccd-c090-4502-bf80-8c70b83a386b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 23 14:59:10.866710 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 14:59:10.866710 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 14:59:10.866710 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 14:59:10.866710 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 14:59:10.866710 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 14:59:10.866710 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 14:59:10.866710 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 14:59:10.866710 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 14:59:10.866710 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 14:59:10.866710 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 14:59:10.866710 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 14:59:10.866710 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 23 14:59:10.866710 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 23 14:59:10.866710 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 23 14:59:10.866710 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 23 14:59:10.867606 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 14:59:10.867606 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 14:59:10.867606 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 23 14:59:10.867606 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 14:59:10.867606 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 23 14:59:10.867606 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 14:59:10.867606 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 14:59:10.867606 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 14:59:10.867606 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 14:59:10.867606 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 14:59:10.867606 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 14:59:10.867606 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application with Lock( Sep 23 14:59:10.867606 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 14:59:10.867606 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._enter() Sep 23 14:59:10.867606 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 14:59:10.867606 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 14:59:10.867606 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 14:59:10.867606 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.creator() Sep 23 14:59:10.869755 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 14:59:10.869755 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application created_value = creator( Sep 23 14:59:10.869755 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 23 14:59:10.869755 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 23 14:59:10.869755 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 23 14:59:10.869755 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 23 14:59:10.869755 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 23 14:59:10.869755 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 23 14:59:10.869755 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 23 14:59:10.869755 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Sep 23 14:59:10.869755 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 240/492] 10.222.0.55 () {56 vars in 1093 bytes} [Fri Sep 23 14:59:10 2022] GET /identity/v3/users/nova => generated 82 bytes in 321 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 14:59:10.869755 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d023bccd-c090-4502-bf80-8c70b83a386b None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:10.900592 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb6c78b-64ea-48ef-988f-d2220bc95ba5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:10.901128 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb6c78b-64ea-48ef-988f-d2220bc95ba5 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:10.901414 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb6c78b-64ea-48ef-988f-d2220bc95ba5 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:10.901718 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb6c78b-64ea-48ef-988f-d2220bc95ba5 None admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:10.902451 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb6c78b-64ea-48ef-988f-d2220bc95ba5 None admin] RBAC: Adding query filter params (name=service) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:59:10.902761 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb6c78b-64ea-48ef-988f-d2220bc95ba5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:10.904183 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb6c78b-64ea-48ef-988f-d2220bc95ba5 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:10.913034 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 253/493] 10.222.0.55 () {56 vars in 1103 bytes} [Fri Sep 23 14:59:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:59:10.913034 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa5f119-51bd-4b5c-8202-878e25b53a00 None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:10.913034 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa5f119-51bd-4b5c-8202-878e25b53a00 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:10.913034 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa5f119-51bd-4b5c-8202-878e25b53a00 None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:10.913034 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 254/494] 10.222.0.55 () {54 vars in 927 bytes} [Fri Sep 23 14:59:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 14:59:10.921199 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa5f119-51bd-4b5c-8202-878e25b53a00 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:10.951655 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:10.978557 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:10.984720 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d023bccd-c090-4502-bf80-8c70b83a386b None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:11.022431 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf16c51-9c2d-4afc-acd8-61035745cd60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:11.023436 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf16c51-9c2d-4afc-acd8-61035745cd60 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:11.023717 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf16c51-9c2d-4afc-acd8-61035745cd60 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:11.023999 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf16c51-9c2d-4afc-acd8-61035745cd60 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:11.033179 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa5f119-51bd-4b5c-8202-878e25b53a00 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:11.036030 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf16c51-9c2d-4afc-acd8-61035745cd60 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 14:59:11.043181 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application [None req-aaf16c51-9c2d-4afc-acd8-61035745cd60 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 23 14:59:11.043181 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 14:59:11.043181 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 14:59:11.043181 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 14:59:11.043181 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 14:59:11.043181 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 14:59:11.043181 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 14:59:11.043181 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 14:59:11.043181 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 14:59:11.043181 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 14:59:11.043181 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 14:59:11.043181 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 14:59:11.043181 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 14:59:11.043181 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 14:59:11.043181 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 14:59:11.043181 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 14:59:11.044138 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 14:59:11.044138 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 14:59:11.044138 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 14:59:11.044138 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 14:59:11.044138 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 14:59:11.044138 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 14:59:11.044138 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 14:59:11.044138 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 14:59:11.044138 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 14:59:11.044138 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 14:59:11.044138 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 14:59:11.044138 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application with Lock( Sep 23 14:59:11.044138 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 14:59:11.044138 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._enter() Sep 23 14:59:11.044138 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 14:59:11.044138 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 14:59:11.044944 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 14:59:11.044944 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.creator() Sep 23 14:59:11.044944 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 14:59:11.044944 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application created_value = creator( Sep 23 14:59:11.044944 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 14:59:11.044944 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 14:59:11.044944 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 14:59:11.044944 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 14:59:11.044944 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 14:59:11.044944 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 14:59:11.044944 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 23 14:59:11.044944 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Sep 23 14:59:11.044944 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 241/495] 10.222.0.55 () {56 vars in 1102 bytes} [Fri Sep 23 14:59:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 14:59:11.045587 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf16c51-9c2d-4afc-acd8-61035745cd60 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:11.081405 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e96120e-bffe-4663-8ba5-04c89e179173 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:11.081600 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e96120e-bffe-4663-8ba5-04c89e179173 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:11.081953 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e96120e-bffe-4663-8ba5-04c89e179173 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:11.082382 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e96120e-bffe-4663-8ba5-04c89e179173 None admin] PATH_INFO: `/v3/users` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:11.083934 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e96120e-bffe-4663-8ba5-04c89e179173 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:59:11.084276 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e96120e-bffe-4663-8ba5-04c89e179173 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:11.085700 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e96120e-bffe-4663-8ba5-04c89e179173 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:11.125357 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 255/496] 10.222.0.55 () {56 vars in 1133 bytes} [Fri Sep 23 14:59:10 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:59:11.127228 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa93d4fe-ebfa-430a-8f9e-ca4a0e96eca2 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:11.127515 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa93d4fe-ebfa-430a-8f9e-ca4a0e96eca2 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:11.127799 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa93d4fe-ebfa-430a-8f9e-ca4a0e96eca2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:11.145312 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-aa93d4fe-ebfa-430a-8f9e-ca4a0e96eca2 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 14:59:11.146504 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-aa93d4fe-ebfa-430a-8f9e-ca4a0e96eca2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 14:59:11.149018 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf16c51-9c2d-4afc-acd8-61035745cd60 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:11.187109 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cadee2-7a5f-457a-864e-bf9b8823c33e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:11.187658 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cadee2-7a5f-457a-864e-bf9b8823c33e None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:11.187937 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cadee2-7a5f-457a-864e-bf9b8823c33e None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:11.188240 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cadee2-7a5f-457a-864e-bf9b8823c33e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:11.198743 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cadee2-7a5f-457a-864e-bf9b8823c33e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:11.200229 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cadee2-7a5f-457a-864e-bf9b8823c33e None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:11.214836 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application [None req-06cadee2-7a5f-457a-864e-bf9b8823c33e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 14:59:11.214836 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 14:59:11.214836 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 14:59:11.214836 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 14:59:11.214836 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 14:59:11.214836 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 14:59:11.214836 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 14:59:11.214836 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 14:59:11.214836 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 14:59:11.214836 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 14:59:11.214836 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 14:59:11.214836 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 14:59:11.214836 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 14:59:11.214836 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 14:59:11.214836 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 14:59:11.214836 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 14:59:11.215887 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 14:59:11.215887 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 14:59:11.215887 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 14:59:11.215887 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 14:59:11.215887 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 14:59:11.215887 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 14:59:11.215887 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 14:59:11.215887 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application with Lock( Sep 23 14:59:11.215887 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 14:59:11.215887 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._enter() Sep 23 14:59:11.215887 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 14:59:11.215887 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 14:59:11.215887 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 14:59:11.215887 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.creator() Sep 23 14:59:11.215887 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 14:59:11.215887 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application created_value = creator( Sep 23 14:59:11.215887 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 14:59:11.216757 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 14:59:11.216757 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 14:59:11.216757 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Sep 23 14:59:11.216757 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 242/497] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Sep 23 14:59:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 14:59:11.216757 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cadee2-7a5f-457a-864e-bf9b8823c33e None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:11.223072 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 256/498] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Sep 23 14:59:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 14:59:11.224422 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ba6f80-2c1f-478c-990e-7a0d2085f9a9 None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:11.224713 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ba6f80-2c1f-478c-990e-7a0d2085f9a9 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:11.224987 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ba6f80-2c1f-478c-990e-7a0d2085f9a9 None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:11.226790 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 257/499] 10.222.0.55 () {54 vars in 927 bytes} [Fri Sep 23 14:59:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 14:59:11.227419 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ba6f80-2c1f-478c-990e-7a0d2085f9a9 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:11.231622 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:11.248804 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:11.329375 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cadee2-7a5f-457a-864e-bf9b8823c33e None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:11.339518 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ba6f80-2c1f-478c-990e-7a0d2085f9a9 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:11.365986 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b091ddb-7094-4214-af51-a7d683015cea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:11.367258 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b091ddb-7094-4214-af51-a7d683015cea None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:11.367742 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b091ddb-7094-4214-af51-a7d683015cea None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:11.368329 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b091ddb-7094-4214-af51-a7d683015cea None admin] PATH_INFO: `/v3/roles` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:11.369258 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b091ddb-7094-4214-af51-a7d683015cea None admin] RBAC: Adding query filter params (name=service) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:59:11.369781 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b091ddb-7094-4214-af51-a7d683015cea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:11.371427 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b091ddb-7094-4214-af51-a7d683015cea None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:11.374076 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7002b46d-3fa9-4b71-8b3c-60ad57502ffc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:11.374697 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7002b46d-3fa9-4b71-8b3c-60ad57502ffc None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:11.374979 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7002b46d-3fa9-4b71-8b3c-60ad57502ffc None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:11.375255 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7002b46d-3fa9-4b71-8b3c-60ad57502ffc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:11.380772 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 243/500] 10.222.0.55 () {56 vars in 1103 bytes} [Fri Sep 23 14:59:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:59:11.381784 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b091ddb-7094-4214-af51-a7d683015cea None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:11.451622 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7002b46d-3fa9-4b71-8b3c-60ad57502ffc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:11.453035 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7002b46d-3fa9-4b71-8b3c-60ad57502ffc None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:11.461446 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:11.476126 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:11.493981 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b091ddb-7094-4214-af51-a7d683015cea None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:11.529200 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9159547c-367f-453a-96fd-bf3f47994c90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:11.529801 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9159547c-367f-453a-96fd-bf3f47994c90 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:11.530078 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9159547c-367f-453a-96fd-bf3f47994c90 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:11.530430 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9159547c-367f-453a-96fd-bf3f47994c90 None admin] PATH_INFO: `/v3/domains` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:11.531118 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9159547c-367f-453a-96fd-bf3f47994c90 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:59:11.531426 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9159547c-367f-453a-96fd-bf3f47994c90 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:11.533075 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9159547c-367f-453a-96fd-bf3f47994c90 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:11.546378 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 244/501] 10.222.0.55 () {56 vars in 1109 bytes} [Fri Sep 23 14:59:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:59:11.548005 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f4edba-1ab9-4829-b67e-67899dd1ddf6 None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:11.548297 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f4edba-1ab9-4829-b67e-67899dd1ddf6 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:11.548848 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f4edba-1ab9-4829-b67e-67899dd1ddf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:11.560284 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 258/502] 10.222.0.55 () {58 vars in 1282 bytes} [Fri Sep 23 14:59:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 23 14:59:11.561114 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7002b46d-3fa9-4b71-8b3c-60ad57502ffc None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:11.570965 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-34f4edba-1ab9-4829-b67e-67899dd1ddf6 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 14:59:11.572886 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-34f4edba-1ab9-4829-b67e-67899dd1ddf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 14:59:11.640166 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 245/503] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Sep 23 14:59:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 14:59:11.641370 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f4edba-1ab9-4829-b67e-67899dd1ddf6 None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:11.669788 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7002b46d-3fa9-4b71-8b3c-60ad57502ffc None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:11.695459 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:11.706841 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:11.718802 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8f2eff-1c41-46f2-abdc-1148eaace640 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:11.720316 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8f2eff-1c41-46f2-abdc-1148eaace640 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:11.720602 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8f2eff-1c41-46f2-abdc-1148eaace640 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:11.720883 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8f2eff-1c41-46f2-abdc-1148eaace640 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:11.733127 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8f2eff-1c41-46f2-abdc-1148eaace640 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:11.734897 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8f2eff-1c41-46f2-abdc-1148eaace640 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:11.751851 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f4edba-1ab9-4829-b67e-67899dd1ddf6 None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:11.752019 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-bf8f2eff-1c41-46f2-abdc-1148eaace640 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 14:59:11.752019 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 14:59:11.752019 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 14:59:11.752019 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 14:59:11.752019 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 14:59:11.752019 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 14:59:11.752019 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 14:59:11.752019 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 14:59:11.752019 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 14:59:11.752019 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 14:59:11.752019 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 14:59:11.752019 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 14:59:11.752019 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 14:59:11.752019 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 14:59:11.752019 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 14:59:11.752019 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 14:59:11.752877 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 14:59:11.752877 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 14:59:11.752877 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 14:59:11.752877 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 14:59:11.752877 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 14:59:11.752877 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 14:59:11.752877 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 14:59:11.752877 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Sep 23 14:59:11.752877 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 14:59:11.752877 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Sep 23 14:59:11.752877 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 14:59:11.752877 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 14:59:11.752877 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 14:59:11.752877 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Sep 23 14:59:11.752877 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 14:59:11.752877 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Sep 23 14:59:11.752877 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 14:59:11.753805 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 14:59:11.753805 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 14:59:11.753805 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Sep 23 14:59:11.757840 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 259/504] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Sep 23 14:59:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 14:59:11.758503 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8f2eff-1c41-46f2-abdc-1148eaace640 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:11.788226 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2e0c98-a52c-4372-9a19-57d531145ca9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:11.788864 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2e0c98-a52c-4372-9a19-57d531145ca9 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:11.789150 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2e0c98-a52c-4372-9a19-57d531145ca9 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:11.789422 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2e0c98-a52c-4372-9a19-57d531145ca9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:11.799894 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2e0c98-a52c-4372-9a19-57d531145ca9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 14:59:11.804737 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application [None req-ec2e0c98-a52c-4372-9a19-57d531145ca9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 23 14:59:11.804737 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 14:59:11.804737 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 14:59:11.804737 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 14:59:11.804737 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 14:59:11.804737 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 14:59:11.804737 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 14:59:11.804737 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 14:59:11.804737 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 14:59:11.804737 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 14:59:11.804737 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 14:59:11.804737 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 14:59:11.804737 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 14:59:11.804737 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 14:59:11.804737 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 14:59:11.804737 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 14:59:11.806201 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 14:59:11.806201 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 14:59:11.806201 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 14:59:11.806201 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 14:59:11.806201 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 14:59:11.806201 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 14:59:11.806201 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 14:59:11.806201 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 14:59:11.806201 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 14:59:11.806201 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 14:59:11.806201 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 14:59:11.806201 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application with Lock( Sep 23 14:59:11.806201 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 14:59:11.806201 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._enter() Sep 23 14:59:11.806201 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 14:59:11.806201 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 14:59:11.807037 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 14:59:11.807037 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.creator() Sep 23 14:59:11.807037 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 14:59:11.807037 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application created_value = creator( Sep 23 14:59:11.807037 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 14:59:11.807037 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 14:59:11.807037 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 14:59:11.807037 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 14:59:11.807037 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 14:59:11.807037 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 14:59:11.807037 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 23 14:59:11.807037 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Sep 23 14:59:11.807037 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 246/505] 10.222.0.55 () {56 vars in 1102 bytes} [Fri Sep 23 14:59:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 14:59:11.807772 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2e0c98-a52c-4372-9a19-57d531145ca9 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:11.876878 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8f2eff-1c41-46f2-abdc-1148eaace640 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:11.911791 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2e0c98-a52c-4372-9a19-57d531145ca9 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:11.914755 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee54eb79-eb6b-4765-b4ee-a0bdcbe21af7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:11.915647 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee54eb79-eb6b-4765-b4ee-a0bdcbe21af7 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:11.916058 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee54eb79-eb6b-4765-b4ee-a0bdcbe21af7 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:11.916363 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee54eb79-eb6b-4765-b4ee-a0bdcbe21af7 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:11.924140 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee54eb79-eb6b-4765-b4ee-a0bdcbe21af7 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:11.925582 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee54eb79-eb6b-4765-b4ee-a0bdcbe21af7 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:11.932501 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:11.939042 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-ee54eb79-eb6b-4765-b4ee-a0bdcbe21af7 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 23 14:59:11.939042 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 14:59:11.939042 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 14:59:11.939042 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 14:59:11.939042 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 14:59:11.939042 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 14:59:11.939042 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 14:59:11.939042 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 14:59:11.939042 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 14:59:11.939042 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 14:59:11.939042 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 14:59:11.939042 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 14:59:11.939042 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 23 14:59:11.939042 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 23 14:59:11.939042 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 23 14:59:11.939042 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 23 14:59:11.940195 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 14:59:11.940195 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 14:59:11.940195 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 23 14:59:11.940195 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 14:59:11.940195 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 23 14:59:11.940195 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 14:59:11.940195 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 14:59:11.940195 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 14:59:11.940195 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 14:59:11.940195 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 14:59:11.940195 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 14:59:11.940195 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Sep 23 14:59:11.940195 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 14:59:11.940195 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Sep 23 14:59:11.940195 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 14:59:11.940195 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 14:59:11.940195 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 14:59:11.940195 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Sep 23 14:59:11.941301 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 14:59:11.941301 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Sep 23 14:59:11.941301 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 23 14:59:11.941301 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 23 14:59:11.941301 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 23 14:59:11.941301 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 23 14:59:11.941301 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 23 14:59:11.941301 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 23 14:59:11.941301 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 23 14:59:11.941301 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Sep 23 14:59:11.941301 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 260/506] 10.222.0.55 () {56 vars in 1099 bytes} [Fri Sep 23 14:59:11 2022] GET /identity/v3/users/glance => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 14:59:11.942026 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee54eb79-eb6b-4765-b4ee-a0bdcbe21af7 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:11.950029 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef4e638-aa9b-426e-a1c4-6ddd18c5755d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:11.951617 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef4e638-aa9b-426e-a1c4-6ddd18c5755d None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:11.951617 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef4e638-aa9b-426e-a1c4-6ddd18c5755d None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:11.951617 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef4e638-aa9b-426e-a1c4-6ddd18c5755d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:11.961827 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef4e638-aa9b-426e-a1c4-6ddd18c5755d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:11.963474 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef4e638-aa9b-426e-a1c4-6ddd18c5755d None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:11.972001 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:11.978280 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application [None req-6ef4e638-aa9b-426e-a1c4-6ddd18c5755d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 14:59:11.978280 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 14:59:11.978280 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 14:59:11.978280 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 14:59:11.978280 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 14:59:11.978280 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 14:59:11.978280 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 14:59:11.978280 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 14:59:11.978280 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 14:59:11.978280 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 14:59:11.978280 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 14:59:11.978280 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 14:59:11.978280 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 14:59:11.978280 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 14:59:11.978280 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 14:59:11.978280 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 14:59:11.979347 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 14:59:11.979347 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 14:59:11.979347 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 14:59:11.979347 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 14:59:11.979347 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 14:59:11.979347 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 14:59:11.979347 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 14:59:11.979347 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application with Lock( Sep 23 14:59:11.979347 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 14:59:11.979347 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._enter() Sep 23 14:59:11.979347 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 14:59:11.979347 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 14:59:11.979347 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 14:59:11.979347 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.creator() Sep 23 14:59:11.979347 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 14:59:11.979347 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application created_value = creator( Sep 23 14:59:11.979347 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 14:59:11.980437 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 14:59:11.980437 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 14:59:11.980437 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Sep 23 14:59:11.980437 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 247/507] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Sep 23 14:59:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 14:59:11.980437 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef4e638-aa9b-426e-a1c4-6ddd18c5755d None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:12.046235 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee54eb79-eb6b-4765-b4ee-a0bdcbe21af7 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:12.083631 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41515e07-48e3-435b-8874-2a81025362e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:12.083947 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef4e638-aa9b-426e-a1c4-6ddd18c5755d None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:12.084278 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41515e07-48e3-435b-8874-2a81025362e8 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:12.084560 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41515e07-48e3-435b-8874-2a81025362e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:12.084832 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41515e07-48e3-435b-8874-2a81025362e8 None admin] PATH_INFO: `/v3/domains` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:12.085479 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41515e07-48e3-435b-8874-2a81025362e8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:59:12.085780 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41515e07-48e3-435b-8874-2a81025362e8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:12.087272 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41515e07-48e3-435b-8874-2a81025362e8 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:12.099442 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 261/508] 10.222.0.55 () {56 vars in 1109 bytes} [Fri Sep 23 14:59:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:59:12.100487 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41515e07-48e3-435b-8874-2a81025362e8 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:12.123408 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ea0cf3-8a70-44a0-8eae-f0dd68623ae5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:12.124040 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ea0cf3-8a70-44a0-8eae-f0dd68623ae5 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:12.124325 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ea0cf3-8a70-44a0-8eae-f0dd68623ae5 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:12.124599 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ea0cf3-8a70-44a0-8eae-f0dd68623ae5 None admin] PATH_INFO: `/v3/roles` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:12.125263 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ea0cf3-8a70-44a0-8eae-f0dd68623ae5 None admin] RBAC: Adding query filter params (name=service) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:59:12.125563 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ea0cf3-8a70-44a0-8eae-f0dd68623ae5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:12.126927 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ea0cf3-8a70-44a0-8eae-f0dd68623ae5 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:12.134832 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 248/509] 10.222.0.55 () {56 vars in 1103 bytes} [Fri Sep 23 14:59:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:59:12.136075 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ea0cf3-8a70-44a0-8eae-f0dd68623ae5 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:12.170666 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:12.191323 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:12.203430 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41515e07-48e3-435b-8874-2a81025362e8 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:12.235459 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ea0cf3-8a70-44a0-8eae-f0dd68623ae5 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:12.237225 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cf7b90-1383-4934-8dee-b2d30aa222b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:12.237864 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cf7b90-1383-4934-8dee-b2d30aa222b3 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:12.238149 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cf7b90-1383-4934-8dee-b2d30aa222b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:12.238425 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cf7b90-1383-4934-8dee-b2d30aa222b3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:12.248614 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cf7b90-1383-4934-8dee-b2d30aa222b3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:12.250044 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cf7b90-1383-4934-8dee-b2d30aa222b3 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:12.266012 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-e2cf7b90-1383-4934-8dee-b2d30aa222b3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 14:59:12.266012 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 14:59:12.266012 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 14:59:12.266012 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 14:59:12.266012 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 14:59:12.266012 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 14:59:12.266012 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 14:59:12.266012 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 14:59:12.266012 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 14:59:12.266012 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 14:59:12.266012 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 14:59:12.266012 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 14:59:12.266012 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 14:59:12.266012 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 14:59:12.266012 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 14:59:12.266012 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 14:59:12.267035 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 14:59:12.267035 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 14:59:12.267035 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 14:59:12.267035 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 14:59:12.267035 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 14:59:12.267035 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 14:59:12.267035 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 14:59:12.267035 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Sep 23 14:59:12.267035 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 14:59:12.267035 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Sep 23 14:59:12.267035 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 14:59:12.267035 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 14:59:12.267035 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 14:59:12.267035 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Sep 23 14:59:12.267035 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 14:59:12.267035 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Sep 23 14:59:12.267035 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 14:59:12.268322 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 14:59:12.268322 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 14:59:12.268322 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Sep 23 14:59:12.268322 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 262/510] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Sep 23 14:59:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 14:59:12.268322 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cf7b90-1383-4934-8dee-b2d30aa222b3 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:12.272047 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efedecb4-a00f-4492-9e5a-cd264374558f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:12.273031 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efedecb4-a00f-4492-9e5a-cd264374558f None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:12.273349 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efedecb4-a00f-4492-9e5a-cd264374558f None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:12.273653 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efedecb4-a00f-4492-9e5a-cd264374558f None admin] PATH_INFO: `/v3/users` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:12.275114 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efedecb4-a00f-4492-9e5a-cd264374558f None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:59:12.275567 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efedecb4-a00f-4492-9e5a-cd264374558f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:12.277216 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efedecb4-a00f-4492-9e5a-cd264374558f None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:12.318598 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 249/511] 10.222.0.55 () {56 vars in 1137 bytes} [Fri Sep 23 14:59:12 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:59:12.319494 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efedecb4-a00f-4492-9e5a-cd264374558f None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:12.392780 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cf7b90-1383-4934-8dee-b2d30aa222b3 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:12.423054 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:12.428091 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80494b38-df57-47fb-b501-a93fdb9e9896 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:12.428701 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80494b38-df57-47fb-b501-a93fdb9e9896 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:12.428982 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80494b38-df57-47fb-b501-a93fdb9e9896 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:12.429262 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80494b38-df57-47fb-b501-a93fdb9e9896 None admin] PATH_INFO: `/v3/domains` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:12.429902 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80494b38-df57-47fb-b501-a93fdb9e9896 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:59:12.430206 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80494b38-df57-47fb-b501-a93fdb9e9896 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:12.431684 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efedecb4-a00f-4492-9e5a-cd264374558f None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:12.432127 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80494b38-df57-47fb-b501-a93fdb9e9896 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:12.441009 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:12.449492 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 263/512] 10.222.0.55 () {56 vars in 1109 bytes} [Fri Sep 23 14:59:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:59:12.449492 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80494b38-df57-47fb-b501-a93fdb9e9896 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:12.468179 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fcaf0b-802c-40ce-a259-97ef9c00b57f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:12.468815 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fcaf0b-802c-40ce-a259-97ef9c00b57f None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:12.469096 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fcaf0b-802c-40ce-a259-97ef9c00b57f None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:12.469371 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fcaf0b-802c-40ce-a259-97ef9c00b57f None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:12.476389 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fcaf0b-802c-40ce-a259-97ef9c00b57f None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:12.478329 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fcaf0b-802c-40ce-a259-97ef9c00b57f None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:12.490371 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application [None req-b2fcaf0b-802c-40ce-a259-97ef9c00b57f None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 23 14:59:12.490371 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 14:59:12.490371 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 14:59:12.490371 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 14:59:12.490371 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 14:59:12.490371 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 14:59:12.490371 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 14:59:12.490371 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 14:59:12.490371 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 14:59:12.490371 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 14:59:12.490371 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 14:59:12.490371 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 14:59:12.490371 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 23 14:59:12.490371 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 23 14:59:12.490371 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 23 14:59:12.490371 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 23 14:59:12.491246 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 14:59:12.491246 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 14:59:12.491246 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 23 14:59:12.491246 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 14:59:12.491246 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 23 14:59:12.491246 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 14:59:12.491246 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 14:59:12.491246 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 14:59:12.491246 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 14:59:12.491246 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 14:59:12.491246 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 14:59:12.491246 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application with Lock( Sep 23 14:59:12.491246 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 14:59:12.491246 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._enter() Sep 23 14:59:12.491246 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 14:59:12.491246 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 14:59:12.491246 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 14:59:12.491246 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.creator() Sep 23 14:59:12.492199 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 14:59:12.492199 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application created_value = creator( Sep 23 14:59:12.492199 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 23 14:59:12.492199 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 23 14:59:12.492199 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 23 14:59:12.492199 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 23 14:59:12.492199 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 23 14:59:12.492199 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 23 14:59:12.492199 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 23 14:59:12.492199 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Sep 23 14:59:12.492199 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 250/513] 10.222.0.55 () {56 vars in 1099 bytes} [Fri Sep 23 14:59:12 2022] GET /identity/v3/users/cinder => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 14:59:12.492199 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fcaf0b-802c-40ce-a259-97ef9c00b57f None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:12.559078 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80494b38-df57-47fb-b501-a93fdb9e9896 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:12.594022 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e145cd98-3d5c-41f7-b4f0-eb1b5ca5cb64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:12.594659 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e145cd98-3d5c-41f7-b4f0-eb1b5ca5cb64 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:12.594941 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e145cd98-3d5c-41f7-b4f0-eb1b5ca5cb64 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:12.595217 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e145cd98-3d5c-41f7-b4f0-eb1b5ca5cb64 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:12.606033 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e145cd98-3d5c-41f7-b4f0-eb1b5ca5cb64 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:12.607506 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e145cd98-3d5c-41f7-b4f0-eb1b5ca5cb64 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:12.609187 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fcaf0b-802c-40ce-a259-97ef9c00b57f None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:12.623086 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-e145cd98-3d5c-41f7-b4f0-eb1b5ca5cb64 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 14:59:12.623086 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 14:59:12.623086 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 14:59:12.623086 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 14:59:12.623086 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 14:59:12.623086 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 14:59:12.623086 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 14:59:12.623086 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 14:59:12.623086 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 14:59:12.623086 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 14:59:12.623086 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 14:59:12.623086 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 14:59:12.623086 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 14:59:12.623086 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 14:59:12.623086 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 14:59:12.623086 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 14:59:12.624188 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 14:59:12.624188 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 14:59:12.624188 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 14:59:12.624188 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 14:59:12.624188 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 14:59:12.624188 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 14:59:12.624188 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 14:59:12.624188 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Sep 23 14:59:12.624188 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 14:59:12.624188 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Sep 23 14:59:12.624188 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 14:59:12.624188 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 14:59:12.624188 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 14:59:12.624188 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Sep 23 14:59:12.624188 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 14:59:12.624188 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Sep 23 14:59:12.624188 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 14:59:12.625058 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 14:59:12.625058 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 14:59:12.625058 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Sep 23 14:59:12.625058 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 264/514] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Sep 23 14:59:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 14:59:12.625464 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e145cd98-3d5c-41f7-b4f0-eb1b5ca5cb64 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:12.648868 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0a6e57-4508-4bc2-a9e1-1cde0082ef2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:12.649637 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0a6e57-4508-4bc2-a9e1-1cde0082ef2a None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:12.650015 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0a6e57-4508-4bc2-a9e1-1cde0082ef2a None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:12.650299 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0a6e57-4508-4bc2-a9e1-1cde0082ef2a None admin] PATH_INFO: `/v3/domains` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:12.650947 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0a6e57-4508-4bc2-a9e1-1cde0082ef2a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:59:12.651255 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0a6e57-4508-4bc2-a9e1-1cde0082ef2a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:12.652673 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0a6e57-4508-4bc2-a9e1-1cde0082ef2a None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:12.665334 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 251/515] 10.222.0.55 () {56 vars in 1109 bytes} [Fri Sep 23 14:59:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:59:12.666241 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0a6e57-4508-4bc2-a9e1-1cde0082ef2a None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:12.681070 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:12.681470 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:12.728576 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e145cd98-3d5c-41f7-b4f0-eb1b5ca5cb64 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:12.762169 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f06e907-3711-4d20-bca1-e2e315b55149 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:12.762699 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f06e907-3711-4d20-bca1-e2e315b55149 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:12.762978 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f06e907-3711-4d20-bca1-e2e315b55149 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:12.763263 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f06e907-3711-4d20-bca1-e2e315b55149 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:12.765773 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0a6e57-4508-4bc2-a9e1-1cde0082ef2a None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:12.800720 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cafe3b-cf36-41e3-b51f-02e2c6cb40cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:12.800720 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cafe3b-cf36-41e3-b51f-02e2c6cb40cf None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:12.800720 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cafe3b-cf36-41e3-b51f-02e2c6cb40cf None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:12.800720 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cafe3b-cf36-41e3-b51f-02e2c6cb40cf None admin] PATH_INFO: `/v3/projects/service` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:12.808282 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5cafe3b-cf36-41e3-b51f-02e2c6cb40cf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:12.809743 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5cafe3b-cf36-41e3-b51f-02e2c6cb40cf None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:12.822714 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application [None req-a5cafe3b-cf36-41e3-b51f-02e2c6cb40cf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 14:59:12.822714 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 14:59:12.822714 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 14:59:12.822714 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 14:59:12.822714 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 14:59:12.822714 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 14:59:12.822714 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 14:59:12.822714 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 14:59:12.822714 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 14:59:12.822714 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 14:59:12.822714 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 14:59:12.822714 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 14:59:12.822714 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 23 14:59:12.822714 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 23 14:59:12.822714 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 23 14:59:12.824172 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 23 14:59:12.824172 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 14:59:12.824172 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 14:59:12.824172 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 14:59:12.824172 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 14:59:12.824172 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 14:59:12.824172 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 14:59:12.824172 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 14:59:12.824172 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application with Lock( Sep 23 14:59:12.824172 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 14:59:12.824172 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._enter() Sep 23 14:59:12.824172 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 14:59:12.824172 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 14:59:12.824172 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 14:59:12.824172 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.creator() Sep 23 14:59:12.824172 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 14:59:12.824172 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application created_value = creator( Sep 23 14:59:12.825026 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 23 14:59:12.825026 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 23 14:59:12.825026 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 23 14:59:12.825026 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 23 14:59:12.825026 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 23 14:59:12.825026 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 23 14:59:12.825026 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 14:59:12.825026 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Sep 23 14:59:12.825026 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 252/516] 10.222.0.55 () {56 vars in 1111 bytes} [Fri Sep 23 14:59:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 14:59:12.825026 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cafe3b-cf36-41e3-b51f-02e2c6cb40cf None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:12.836545 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f06e907-3711-4d20-bca1-e2e315b55149 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:12.838600 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f06e907-3711-4d20-bca1-e2e315b55149 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:12.895261 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:12.895485 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:12.925846 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cafe3b-cf36-41e3-b51f-02e2c6cb40cf None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:12.947026 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 265/517] 10.222.0.55 () {58 vars in 1282 bytes} [Fri Sep 23 14:59:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 23 14:59:12.947026 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f06e907-3711-4d20-bca1-e2e315b55149 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:12.966617 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7e30b9-74c8-40c2-ba02-8a07c438d0d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:12.967233 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7e30b9-74c8-40c2-ba02-8a07c438d0d3 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:12.967516 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7e30b9-74c8-40c2-ba02-8a07c438d0d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:12.967793 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7e30b9-74c8-40c2-ba02-8a07c438d0d3 None admin] PATH_INFO: `/v3/users` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:12.968565 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7e30b9-74c8-40c2-ba02-8a07c438d0d3 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:59:12.968871 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7e30b9-74c8-40c2-ba02-8a07c438d0d3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:12.970372 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7e30b9-74c8-40c2-ba02-8a07c438d0d3 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:12.988177 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 253/518] 10.222.0.55 () {56 vars in 1137 bytes} [Fri Sep 23 14:59:12 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:59:12.988968 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7e30b9-74c8-40c2-ba02-8a07c438d0d3 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:13.054562 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f06e907-3711-4d20-bca1-e2e315b55149 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:13.089210 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7e30b9-74c8-40c2-ba02-8a07c438d0d3 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:13.090375 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2447da11-b20e-4c3f-ae51-e42a0736328f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:13.090988 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2447da11-b20e-4c3f-ae51-e42a0736328f None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:13.091267 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2447da11-b20e-4c3f-ae51-e42a0736328f None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:13.091545 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2447da11-b20e-4c3f-ae51-e42a0736328f None admin] PATH_INFO: `/v3/domains` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:13.092221 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2447da11-b20e-4c3f-ae51-e42a0736328f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:59:13.092593 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2447da11-b20e-4c3f-ae51-e42a0736328f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:13.093933 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2447da11-b20e-4c3f-ae51-e42a0736328f None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:13.105594 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 266/519] 10.222.0.55 () {56 vars in 1109 bytes} [Fri Sep 23 14:59:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:59:13.106393 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2447da11-b20e-4c3f-ae51-e42a0736328f None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:13.121077 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:13.121616 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:13.127199 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3070b2-f6d2-4841-abb2-6dbc4d86b626 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:13.127850 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3070b2-f6d2-4841-abb2-6dbc4d86b626 None admin] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:13.128107 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3070b2-f6d2-4841-abb2-6dbc4d86b626 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:13.128377 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3070b2-f6d2-4841-abb2-6dbc4d86b626 None admin] PATH_INFO: `/v3/users` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:13.129008 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3070b2-f6d2-4841-abb2-6dbc4d86b626 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:13.139893 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3070b2-f6d2-4841-abb2-6dbc4d86b626 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:13.204780 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2447da11-b20e-4c3f-ae51-e42a0736328f None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:13.237641 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc63517a-eeef-4b1f-9641-8f7d93457084 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:13.238249 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc63517a-eeef-4b1f-9641-8f7d93457084 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:13.238532 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc63517a-eeef-4b1f-9641-8f7d93457084 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:13.238810 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc63517a-eeef-4b1f-9641-8f7d93457084 None admin] PATH_INFO: `/v3/projects` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:13.239494 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc63517a-eeef-4b1f-9641-8f7d93457084 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:59:13.239802 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc63517a-eeef-4b1f-9641-8f7d93457084 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:13.241261 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc63517a-eeef-4b1f-9641-8f7d93457084 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:13.259667 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 267/520] 10.222.0.55 () {56 vars in 1148 bytes} [Fri Sep 23 14:59:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:59:13.260493 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc63517a-eeef-4b1f-9641-8f7d93457084 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:13.291475 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 254/521] 10.222.0.55 () {60 vars in 1132 bytes} [Fri Sep 23 14:59:12 2022] POST /identity/v3/users => generated 247 bytes in 303 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 14:59:13.292259 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3070b2-f6d2-4841-abb2-6dbc4d86b626 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:13.336766 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:13.372499 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc63517a-eeef-4b1f-9641-8f7d93457084 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:13.411377 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3070b2-f6d2-4841-abb2-6dbc4d86b626 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:13.415121 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f49e06a-69bc-4d81-8d7a-e38088731669 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:13.415782 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f49e06a-69bc-4d81-8d7a-e38088731669 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:13.416099 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f49e06a-69bc-4d81-8d7a-e38088731669 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:13.416378 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f49e06a-69bc-4d81-8d7a-e38088731669 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:13.429206 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f49e06a-69bc-4d81-8d7a-e38088731669 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:13.431292 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f49e06a-69bc-4d81-8d7a-e38088731669 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:13.448978 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b5efd1-2136-48f3-be56-3b4740b473bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:13.449691 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b5efd1-2136-48f3-be56-3b4740b473bc None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:13.449885 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b5efd1-2136-48f3-be56-3b4740b473bc None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:13.450161 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b5efd1-2136-48f3-be56-3b4740b473bc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:13.450397 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-8f49e06a-69bc-4d81-8d7a-e38088731669 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 14:59:13.450397 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 14:59:13.450397 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 14:59:13.450397 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 14:59:13.450397 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 14:59:13.450397 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 14:59:13.450397 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 14:59:13.450397 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 14:59:13.450397 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 14:59:13.450397 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 14:59:13.450397 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 14:59:13.450397 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 14:59:13.450397 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 14:59:13.450397 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 14:59:13.450397 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 14:59:13.450397 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 14:59:13.451252 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 14:59:13.451252 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 14:59:13.451252 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 14:59:13.451252 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 14:59:13.451252 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 14:59:13.451252 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 14:59:13.451252 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 14:59:13.451252 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Sep 23 14:59:13.451252 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 14:59:13.451252 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Sep 23 14:59:13.451252 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 14:59:13.451252 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 14:59:13.451252 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 14:59:13.451252 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Sep 23 14:59:13.451252 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 14:59:13.451252 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Sep 23 14:59:13.451252 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 14:59:13.452159 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 14:59:13.452159 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 14:59:13.452159 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Sep 23 14:59:13.452159 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 268/522] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Sep 23 14:59:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 14:59:13.452384 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f49e06a-69bc-4d81-8d7a-e38088731669 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:13.522927 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:13.537204 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b5efd1-2136-48f3-be56-3b4740b473bc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:13.538680 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b5efd1-2136-48f3-be56-3b4740b473bc None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:13.572219 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f49e06a-69bc-4d81-8d7a-e38088731669 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:13.597317 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:13.609002 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d918805-f952-44e5-80da-745ae49c17eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:13.609669 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d918805-f952-44e5-80da-745ae49c17eb None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:13.609945 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d918805-f952-44e5-80da-745ae49c17eb None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:13.610221 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d918805-f952-44e5-80da-745ae49c17eb None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:13.620409 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d918805-f952-44e5-80da-745ae49c17eb None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:13.621983 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d918805-f952-44e5-80da-745ae49c17eb None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:13.636110 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-9d918805-f952-44e5-80da-745ae49c17eb None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 23 14:59:13.636110 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 14:59:13.636110 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 14:59:13.636110 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 14:59:13.636110 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 14:59:13.636110 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 14:59:13.636110 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 14:59:13.636110 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 14:59:13.636110 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 14:59:13.636110 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 14:59:13.636110 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 14:59:13.636110 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 14:59:13.636110 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 23 14:59:13.636110 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 23 14:59:13.636110 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 23 14:59:13.636110 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 23 14:59:13.637263 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 14:59:13.637263 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 14:59:13.637263 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 23 14:59:13.637263 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 14:59:13.637263 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 23 14:59:13.637263 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 14:59:13.637263 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 14:59:13.637263 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 14:59:13.637263 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 14:59:13.637263 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 14:59:13.637263 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 14:59:13.637263 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Sep 23 14:59:13.637263 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 14:59:13.637263 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Sep 23 14:59:13.637263 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 14:59:13.637263 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 14:59:13.637263 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 14:59:13.637263 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Sep 23 14:59:13.638387 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 14:59:13.638387 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Sep 23 14:59:13.638387 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 23 14:59:13.638387 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 23 14:59:13.638387 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 23 14:59:13.638387 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 23 14:59:13.638387 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 23 14:59:13.638387 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 23 14:59:13.638387 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 23 14:59:13.638387 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Sep 23 14:59:13.644273 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 269/523] 10.222.0.55 () {56 vars in 1102 bytes} [Fri Sep 23 14:59:13 2022] GET /identity/v3/users/neutron => generated 85 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 14:59:13.644273 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d918805-f952-44e5-80da-745ae49c17eb None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:13.660195 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 255/524] 10.222.0.55 () {58 vars in 1282 bytes} [Fri Sep 23 14:59:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 23 14:59:13.661305 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b5efd1-2136-48f3-be56-3b4740b473bc None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:13.754765 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:13.759510 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d918805-f952-44e5-80da-745ae49c17eb None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:13.775660 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b5efd1-2136-48f3-be56-3b4740b473bc None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:13.795260 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f5f612-e5d3-4399-9f9d-e1f97d31843e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:13.795871 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f5f612-e5d3-4399-9f9d-e1f97d31843e None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:13.796178 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f5f612-e5d3-4399-9f9d-e1f97d31843e None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:13.796450 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f5f612-e5d3-4399-9f9d-e1f97d31843e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:13.797170 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f5f612-e5d3-4399-9f9d-e1f97d31843e None admin] RBAC: Adding query filter params (role.id=0665869b71fa40b0b552efdd57faf023, scope.project.id=87b3e9d4b6474bd4a04ba59ef52bf3e2, user.id=b3c6c7a1325a47be924268e1aa1a7a18) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:59:13.797478 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f5f612-e5d3-4399-9f9d-e1f97d31843e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:13.798903 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f5f612-e5d3-4399-9f9d-e1f97d31843e None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:13.804585 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 270/525] 10.222.0.55 () {56 vars in 1374 bytes} [Fri Sep 23 14:59:13 2022] GET /identity/v3/role_assignments?user.id=b3c6c7a1325a47be924268e1aa1a7a18&scope.project.id=87b3e9d4b6474bd4a04ba59ef52bf3e2&role.id=0665869b71fa40b0b552efdd57faf023 => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:59:13.805381 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f5f612-e5d3-4399-9f9d-e1f97d31843e None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:13.827374 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19683a3e-03d0-481a-adc5-13256d330004 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:13.828767 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19683a3e-03d0-481a-adc5-13256d330004 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:13.829239 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19683a3e-03d0-481a-adc5-13256d330004 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:13.829698 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19683a3e-03d0-481a-adc5-13256d330004 None admin] PATH_INFO: `/v3/domains` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:13.830835 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19683a3e-03d0-481a-adc5-13256d330004 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:59:13.831887 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19683a3e-03d0-481a-adc5-13256d330004 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:13.833894 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19683a3e-03d0-481a-adc5-13256d330004 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:13.851771 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 256/526] 10.222.0.55 () {56 vars in 1109 bytes} [Fri Sep 23 14:59:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:59:13.853426 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19683a3e-03d0-481a-adc5-13256d330004 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:13.878484 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:13.952322 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f5f612-e5d3-4399-9f9d-e1f97d31843e None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:13.988794 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19683a3e-03d0-481a-adc5-13256d330004 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:13.992829 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0a6e57-bfaa-4f2c-bb51-a3dc0c6c1e11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:13.993612 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0a6e57-bfaa-4f2c-bb51-a3dc0c6c1e11 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:13.993901 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0a6e57-bfaa-4f2c-bb51-a3dc0c6c1e11 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:13.994175 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0a6e57-bfaa-4f2c-bb51-a3dc0c6c1e11 None admin] PATH_INFO: `/v3/users` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:13.994920 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0a6e57-bfaa-4f2c-bb51-a3dc0c6c1e11 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:59:13.995225 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0a6e57-bfaa-4f2c-bb51-a3dc0c6c1e11 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:13.996728 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0a6e57-bfaa-4f2c-bb51-a3dc0c6c1e11 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:14.017292 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 271/527] 10.222.0.55 () {56 vars in 1139 bytes} [Fri Sep 23 14:59:13 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:59:14.018237 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0a6e57-bfaa-4f2c-bb51-a3dc0c6c1e11 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:14.034046 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755c215f-a665-4c69-9a76-91c744f7c998 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:14.035158 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755c215f-a665-4c69-9a76-91c744f7c998 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:14.035793 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755c215f-a665-4c69-9a76-91c744f7c998 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:14.036204 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755c215f-a665-4c69-9a76-91c744f7c998 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:14.053770 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755c215f-a665-4c69-9a76-91c744f7c998 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:14.056823 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755c215f-a665-4c69-9a76-91c744f7c998 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:14.067239 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:14.078796 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application [None req-755c215f-a665-4c69-9a76-91c744f7c998 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 14:59:14.078796 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 14:59:14.078796 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 14:59:14.078796 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 14:59:14.078796 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 14:59:14.078796 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 14:59:14.078796 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 14:59:14.078796 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 14:59:14.078796 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 14:59:14.078796 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 14:59:14.078796 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 14:59:14.078796 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 14:59:14.078796 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 14:59:14.078796 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 14:59:14.078796 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 14:59:14.078796 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 14:59:14.085810 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 14:59:14.085810 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 14:59:14.085810 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 14:59:14.085810 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 14:59:14.085810 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 14:59:14.085810 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 14:59:14.085810 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 14:59:14.085810 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application with Lock( Sep 23 14:59:14.085810 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 14:59:14.085810 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._enter() Sep 23 14:59:14.085810 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 14:59:14.085810 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 14:59:14.085810 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 14:59:14.085810 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.creator() Sep 23 14:59:14.085810 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 14:59:14.085810 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application created_value = creator( Sep 23 14:59:14.085810 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 14:59:14.088376 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 14:59:14.088376 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 14:59:14.088376 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Sep 23 14:59:14.088376 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 257/528] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Sep 23 14:59:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 14:59:14.088376 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755c215f-a665-4c69-9a76-91c744f7c998 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:14.156031 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0a6e57-bfaa-4f2c-bb51-a3dc0c6c1e11 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:14.164102 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:14.197298 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f427f6f0-3b7b-478d-8d1e-a867d812c868 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:14.197928 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f427f6f0-3b7b-478d-8d1e-a867d812c868 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:14.198205 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f427f6f0-3b7b-478d-8d1e-a867d812c868 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:14.198476 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f427f6f0-3b7b-478d-8d1e-a867d812c868 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:14.207171 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f427f6f0-3b7b-478d-8d1e-a867d812c868 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:14.208766 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f427f6f0-3b7b-478d-8d1e-a867d812c868 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:14.219372 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755c215f-a665-4c69-9a76-91c744f7c998 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:14.223379 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-f427f6f0-3b7b-478d-8d1e-a867d812c868 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 14:59:14.223379 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 14:59:14.223379 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 14:59:14.223379 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 14:59:14.223379 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 14:59:14.223379 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 14:59:14.223379 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 14:59:14.223379 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 14:59:14.223379 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 14:59:14.223379 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 14:59:14.223379 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 14:59:14.223379 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 14:59:14.223379 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 23 14:59:14.223379 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 23 14:59:14.223379 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 23 14:59:14.224345 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 23 14:59:14.224345 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 14:59:14.224345 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 14:59:14.224345 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 14:59:14.224345 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 14:59:14.224345 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 14:59:14.224345 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 14:59:14.224345 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 14:59:14.224345 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Sep 23 14:59:14.224345 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 14:59:14.224345 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Sep 23 14:59:14.224345 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 14:59:14.224345 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 14:59:14.224345 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 14:59:14.224345 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Sep 23 14:59:14.224345 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 14:59:14.224345 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Sep 23 14:59:14.225213 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 23 14:59:14.225213 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 23 14:59:14.225213 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 23 14:59:14.225213 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 23 14:59:14.225213 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 23 14:59:14.225213 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 23 14:59:14.225213 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 14:59:14.225213 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Sep 23 14:59:14.225213 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 272/529] 10.222.0.55 () {56 vars in 1111 bytes} [Fri Sep 23 14:59:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 14:59:14.225676 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f427f6f0-3b7b-478d-8d1e-a867d812c868 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:14.263933 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1287e7bf-4100-4464-947c-706cbf6f8cef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:14.264883 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1287e7bf-4100-4464-947c-706cbf6f8cef None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:14.265308 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1287e7bf-4100-4464-947c-706cbf6f8cef None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:14.265728 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1287e7bf-4100-4464-947c-706cbf6f8cef None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:14.338005 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:14.342305 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1287e7bf-4100-4464-947c-706cbf6f8cef None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:14.343935 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1287e7bf-4100-4464-947c-706cbf6f8cef None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:14.356832 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f427f6f0-3b7b-478d-8d1e-a867d812c868 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:14.397944 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fe87a3-b0d3-4e17-9222-5398a1e0d201 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:14.399224 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fe87a3-b0d3-4e17-9222-5398a1e0d201 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:14.399810 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fe87a3-b0d3-4e17-9222-5398a1e0d201 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:14.400268 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fe87a3-b0d3-4e17-9222-5398a1e0d201 None admin] PATH_INFO: `/v3/domains` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:14.401054 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fe87a3-b0d3-4e17-9222-5398a1e0d201 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:59:14.401500 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fe87a3-b0d3-4e17-9222-5398a1e0d201 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:14.403018 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fe87a3-b0d3-4e17-9222-5398a1e0d201 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:14.418276 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 273/530] 10.222.0.55 () {56 vars in 1109 bytes} [Fri Sep 23 14:59:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:59:14.419728 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fe87a3-b0d3-4e17-9222-5398a1e0d201 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:14.451810 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:14.478871 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 258/531] 10.222.0.55 () {58 vars in 1282 bytes} [Fri Sep 23 14:59:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 23 14:59:14.479915 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1287e7bf-4100-4464-947c-706cbf6f8cef None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:14.547436 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fe87a3-b0d3-4e17-9222-5398a1e0d201 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:14.588447 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:14.591884 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1287e7bf-4100-4464-947c-706cbf6f8cef None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:14.592045 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d7865d-eb49-4356-91db-bdbe254d64a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:14.592568 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d7865d-eb49-4356-91db-bdbe254d64a0 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:14.592846 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d7865d-eb49-4356-91db-bdbe254d64a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:14.593121 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d7865d-eb49-4356-91db-bdbe254d64a0 None admin] PATH_INFO: `/v3/projects` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:14.593815 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d7865d-eb49-4356-91db-bdbe254d64a0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:59:14.594118 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d7865d-eb49-4356-91db-bdbe254d64a0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:14.596313 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d7865d-eb49-4356-91db-bdbe254d64a0 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:14.614303 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 274/532] 10.222.0.55 () {56 vars in 1148 bytes} [Fri Sep 23 14:59:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:59:14.615565 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d7865d-eb49-4356-91db-bdbe254d64a0 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:14.635040 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caaa27fe-c63c-4ca6-b212-ac374a09fcd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:14.635508 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaa27fe-c63c-4ca6-b212-ac374a09fcd0 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:14.635962 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaa27fe-c63c-4ca6-b212-ac374a09fcd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:14.636313 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaa27fe-c63c-4ca6-b212-ac374a09fcd0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:14.644023 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caaa27fe-c63c-4ca6-b212-ac374a09fcd0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:14.645472 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caaa27fe-c63c-4ca6-b212-ac374a09fcd0 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:14.659078 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application [None req-caaa27fe-c63c-4ca6-b212-ac374a09fcd0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 14:59:14.659078 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 14:59:14.659078 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 14:59:14.659078 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 14:59:14.659078 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 14:59:14.659078 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 14:59:14.659078 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 14:59:14.659078 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 14:59:14.659078 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 14:59:14.659078 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 14:59:14.659078 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 14:59:14.659078 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 14:59:14.659078 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 23 14:59:14.659078 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 23 14:59:14.659078 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 23 14:59:14.659971 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 23 14:59:14.659971 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 14:59:14.659971 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 14:59:14.659971 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 14:59:14.659971 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 14:59:14.659971 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 14:59:14.659971 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 14:59:14.659971 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 14:59:14.659971 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application with Lock( Sep 23 14:59:14.659971 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 14:59:14.659971 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._enter() Sep 23 14:59:14.659971 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 14:59:14.659971 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 14:59:14.659971 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 14:59:14.659971 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.creator() Sep 23 14:59:14.659971 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 14:59:14.659971 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application created_value = creator( Sep 23 14:59:14.661268 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 23 14:59:14.661268 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 23 14:59:14.661268 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 23 14:59:14.661268 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 23 14:59:14.661268 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 23 14:59:14.661268 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 23 14:59:14.661268 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 14:59:14.661268 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Sep 23 14:59:14.661268 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 259/533] 10.222.0.55 () {56 vars in 1111 bytes} [Fri Sep 23 14:59:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 14:59:14.661268 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caaa27fe-c63c-4ca6-b212-ac374a09fcd0 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:14.718734 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:14.735462 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d7865d-eb49-4356-91db-bdbe254d64a0 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:14.777857 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf4a866-40c1-48c7-89ba-8a83a81c39a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:14.779019 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf4a866-40c1-48c7-89ba-8a83a81c39a2 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:14.779551 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf4a866-40c1-48c7-89ba-8a83a81c39a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:14.780050 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf4a866-40c1-48c7-89ba-8a83a81c39a2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:14.782528 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caaa27fe-c63c-4ca6-b212-ac374a09fcd0 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:14.791937 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf4a866-40c1-48c7-89ba-8a83a81c39a2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:14.794140 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf4a866-40c1-48c7-89ba-8a83a81c39a2 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:14.812239 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-ecf4a866-40c1-48c7-89ba-8a83a81c39a2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 14:59:14.812239 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 14:59:14.812239 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 14:59:14.812239 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 14:59:14.812239 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 14:59:14.812239 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 14:59:14.812239 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 14:59:14.812239 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 14:59:14.812239 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 14:59:14.812239 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 14:59:14.812239 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 14:59:14.812239 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 14:59:14.812239 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 14:59:14.812239 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 14:59:14.812239 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 14:59:14.812239 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 14:59:14.813435 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 14:59:14.813435 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 14:59:14.813435 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 14:59:14.813435 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 14:59:14.813435 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 14:59:14.813435 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 14:59:14.813435 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 14:59:14.813435 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Sep 23 14:59:14.813435 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 14:59:14.813435 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Sep 23 14:59:14.813435 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 14:59:14.813435 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 14:59:14.813435 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 14:59:14.813435 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Sep 23 14:59:14.813435 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 14:59:14.813435 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Sep 23 14:59:14.813435 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 14:59:14.814376 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 14:59:14.814376 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 14:59:14.814376 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Sep 23 14:59:14.816628 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 275/534] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Sep 23 14:59:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 14:59:14.817909 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf4a866-40c1-48c7-89ba-8a83a81c39a2 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:14.820454 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdf32ce-fdee-43a5-8dc9-b24dfe35a42b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:14.822314 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdf32ce-fdee-43a5-8dc9-b24dfe35a42b None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:14.822595 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdf32ce-fdee-43a5-8dc9-b24dfe35a42b None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:14.822869 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdf32ce-fdee-43a5-8dc9-b24dfe35a42b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:14.824062 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdf32ce-fdee-43a5-8dc9-b24dfe35a42b None admin] RBAC: Adding query filter params (role.id=0665869b71fa40b0b552efdd57faf023, scope.project.id=87b3e9d4b6474bd4a04ba59ef52bf3e2, user.id=51c39f1d05634589a88c0b6477bd014e) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:59:14.824173 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdf32ce-fdee-43a5-8dc9-b24dfe35a42b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:14.825738 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdf32ce-fdee-43a5-8dc9-b24dfe35a42b None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:14.832584 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 260/535] 10.222.0.55 () {56 vars in 1374 bytes} [Fri Sep 23 14:59:14 2022] GET /identity/v3/role_assignments?user.id=51c39f1d05634589a88c0b6477bd014e&scope.project.id=87b3e9d4b6474bd4a04ba59ef52bf3e2&role.id=0665869b71fa40b0b552efdd57faf023 => generated 259 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:59:14.833279 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdf32ce-fdee-43a5-8dc9-b24dfe35a42b None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:14.861969 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:14.934200 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf4a866-40c1-48c7-89ba-8a83a81c39a2 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:14.949711 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdf32ce-fdee-43a5-8dc9-b24dfe35a42b None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:14.975839 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19279be4-c7e3-4616-8ef7-761e623493a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:14.976950 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19279be4-c7e3-4616-8ef7-761e623493a2 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:14.977437 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19279be4-c7e3-4616-8ef7-761e623493a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:14.977946 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19279be4-c7e3-4616-8ef7-761e623493a2 None admin] PATH_INFO: `/v3/projects` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:14.979072 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19279be4-c7e3-4616-8ef7-761e623493a2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:59:14.979707 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19279be4-c7e3-4616-8ef7-761e623493a2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:14.981457 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19279be4-c7e3-4616-8ef7-761e623493a2 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:14.988329 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f82aec5-33d5-41e2-ae92-fb1cc02bf530 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:14.988993 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f82aec5-33d5-41e2-ae92-fb1cc02bf530 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:14.989270 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f82aec5-33d5-41e2-ae92-fb1cc02bf530 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:14.989540 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f82aec5-33d5-41e2-ae92-fb1cc02bf530 None admin] PATH_INFO: `/v3/domains` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:14.990179 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f82aec5-33d5-41e2-ae92-fb1cc02bf530 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:59:14.990476 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f82aec5-33d5-41e2-ae92-fb1cc02bf530 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:14.991841 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:14.993036 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f82aec5-33d5-41e2-ae92-fb1cc02bf530 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:14.998806 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 276/536] 10.222.0.55 () {56 vars in 1148 bytes} [Fri Sep 23 14:59:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:59:15.012586 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 261/537] 10.222.0.55 () {56 vars in 1109 bytes} [Fri Sep 23 14:59:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:59:15.017042 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19279be4-c7e3-4616-8ef7-761e623493a2 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:15.018071 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f82aec5-33d5-41e2-ae92-fb1cc02bf530 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:15.172358 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19279be4-c7e3-4616-8ef7-761e623493a2 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:15.175175 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:15.200754 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f82aec5-33d5-41e2-ae92-fb1cc02bf530 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:15.246111 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad7a80d-0291-4a3e-8357-7d2f128b7951 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:15.247111 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad7a80d-0291-4a3e-8357-7d2f128b7951 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:15.247588 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad7a80d-0291-4a3e-8357-7d2f128b7951 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:15.248099 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad7a80d-0291-4a3e-8357-7d2f128b7951 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:15.249022 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad7a80d-0291-4a3e-8357-7d2f128b7951 None admin] RBAC: Adding query filter params (role.id=0665869b71fa40b0b552efdd57faf023, scope.project.id=87b3e9d4b6474bd4a04ba59ef52bf3e2, user.id=feca6278be904ef8b0dc580a67e477f8) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:59:15.249526 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad7a80d-0291-4a3e-8357-7d2f128b7951 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:15.251138 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad7a80d-0291-4a3e-8357-7d2f128b7951 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:15.263400 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 277/538] 10.222.0.55 () {56 vars in 1374 bytes} [Fri Sep 23 14:59:15 2022] GET /identity/v3/role_assignments?user.id=feca6278be904ef8b0dc580a67e477f8&scope.project.id=87b3e9d4b6474bd4a04ba59ef52bf3e2&role.id=0665869b71fa40b0b552efdd57faf023 => generated 259 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:59:15.279768 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9b2119-edda-41fd-abb2-8a725ae5e40f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:15.280807 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9b2119-edda-41fd-abb2-8a725ae5e40f None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:15.281277 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9b2119-edda-41fd-abb2-8a725ae5e40f None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:15.281737 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9b2119-edda-41fd-abb2-8a725ae5e40f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:15.294984 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9b2119-edda-41fd-abb2-8a725ae5e40f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:15.296854 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9b2119-edda-41fd-abb2-8a725ae5e40f None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:15.311967 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application [None req-9e9b2119-edda-41fd-abb2-8a725ae5e40f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 14:59:15.311967 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 14:59:15.311967 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 14:59:15.311967 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 14:59:15.311967 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 14:59:15.311967 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 14:59:15.311967 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 14:59:15.311967 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 14:59:15.311967 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 14:59:15.311967 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 14:59:15.311967 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 14:59:15.311967 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 14:59:15.311967 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 23 14:59:15.311967 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 23 14:59:15.311967 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 23 14:59:15.312885 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 23 14:59:15.312885 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 14:59:15.312885 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 14:59:15.312885 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 14:59:15.312885 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 14:59:15.312885 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 14:59:15.312885 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 14:59:15.312885 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 14:59:15.312885 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application with Lock( Sep 23 14:59:15.312885 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 14:59:15.312885 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._enter() Sep 23 14:59:15.312885 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 14:59:15.312885 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 14:59:15.312885 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 14:59:15.312885 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.creator() Sep 23 14:59:15.312885 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 14:59:15.312885 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application created_value = creator( Sep 23 14:59:15.313730 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 23 14:59:15.313730 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 23 14:59:15.313730 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 23 14:59:15.313730 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 23 14:59:15.313730 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 23 14:59:15.313730 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 23 14:59:15.313730 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 14:59:15.313730 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Sep 23 14:59:15.315854 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 262/539] 10.222.0.55 () {56 vars in 1111 bytes} [Fri Sep 23 14:59:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 14:59:15.322757 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad7a80d-0291-4a3e-8357-7d2f128b7951 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:15.426654 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:15.495628 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad7a80d-0291-4a3e-8357-7d2f128b7951 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:15.534252 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef642df5-97fe-4e27-a8f8-9967b5bfa58c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:15.535208 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef642df5-97fe-4e27-a8f8-9967b5bfa58c None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:15.535847 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef642df5-97fe-4e27-a8f8-9967b5bfa58c None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:15.536340 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef642df5-97fe-4e27-a8f8-9967b5bfa58c None admin] PATH_INFO: `/v3/projects` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:15.537223 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef642df5-97fe-4e27-a8f8-9967b5bfa58c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:59:15.537721 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef642df5-97fe-4e27-a8f8-9967b5bfa58c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:15.539343 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef642df5-97fe-4e27-a8f8-9967b5bfa58c None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:15.552356 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 278/540] 10.222.0.55 () {56 vars in 1148 bytes} [Fri Sep 23 14:59:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:59:15.560706 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9b2119-edda-41fd-abb2-8a725ae5e40f None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:15.693156 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958eed59-f02c-4089-8949-f7378cfea627 None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:15.694249 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958eed59-f02c-4089-8949-f7378cfea627 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:15.695602 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958eed59-f02c-4089-8949-f7378cfea627 None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:15.699308 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 279/541] 10.222.0.55 () {54 vars in 927 bytes} [Fri Sep 23 14:59:15 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 14:59:15.709974 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0657ad-8145-4ff8-a327-20600bb48245 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:15.714068 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0657ad-8145-4ff8-a327-20600bb48245 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:15.714068 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0657ad-8145-4ff8-a327-20600bb48245 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:15.775599 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-7e0657ad-8145-4ff8-a327-20600bb48245 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 14:59:15.777057 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-7e0657ad-8145-4ff8-a327-20600bb48245 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 14:59:15.793605 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9b2119-edda-41fd-abb2-8a725ae5e40f None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:15.863405 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ac038e-f760-4c65-a4d6-9e7763fd6388 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:15.864045 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ac038e-f760-4c65-a4d6-9e7763fd6388 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:15.864317 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ac038e-f760-4c65-a4d6-9e7763fd6388 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:15.864584 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ac038e-f760-4c65-a4d6-9e7763fd6388 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:15.865307 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ac038e-f760-4c65-a4d6-9e7763fd6388 None admin] RBAC: Adding query filter params (role.id=0665869b71fa40b0b552efdd57faf023, scope.project.id=87b3e9d4b6474bd4a04ba59ef52bf3e2, user.id=d259178570864ffeb936ee10dfec6b28) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:59:15.865611 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ac038e-f760-4c65-a4d6-9e7763fd6388 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:15.867048 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ac038e-f760-4c65-a4d6-9e7763fd6388 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:15.881168 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 263/542] 10.222.0.55 () {56 vars in 1374 bytes} [Fri Sep 23 14:59:15 2022] GET /identity/v3/role_assignments?user.id=d259178570864ffeb936ee10dfec6b28&scope.project.id=87b3e9d4b6474bd4a04ba59ef52bf3e2&role.id=0665869b71fa40b0b552efdd57faf023 => generated 259 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:59:15.917016 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 280/543] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Sep 23 14:59:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 14:59:16.335134 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ac038e-f760-4c65-a4d6-9e7763fd6388 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:16.344843 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:16.405469 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9c32ab-cc61-4044-a3e5-cfa821ed30b8 None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:16.405782 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9c32ab-cc61-4044-a3e5-cfa821ed30b8 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:16.406026 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9c32ab-cc61-4044-a3e5-cfa821ed30b8 None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:16.407881 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 281/544] 10.222.0.55 () {54 vars in 927 bytes} [Fri Sep 23 14:59:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 14:59:16.413273 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0100b577-f848-4485-9800-2f3dd3deec1d None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:16.413562 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0100b577-f848-4485-9800-2f3dd3deec1d None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:16.413838 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0100b577-f848-4485-9800-2f3dd3deec1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:16.444774 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-0100b577-f848-4485-9800-2f3dd3deec1d None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 14:59:16.446001 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-0100b577-f848-4485-9800-2f3dd3deec1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 14:59:16.465529 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ac038e-f760-4c65-a4d6-9e7763fd6388 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:16.504057 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e45a90d-8de9-4fd1-b62f-f05f9a246161 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:16.504842 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e45a90d-8de9-4fd1-b62f-f05f9a246161 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:16.505150 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e45a90d-8de9-4fd1-b62f-f05f9a246161 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:16.505427 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e45a90d-8de9-4fd1-b62f-f05f9a246161 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:16.518548 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e45a90d-8de9-4fd1-b62f-f05f9a246161 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 14:59:16.524866 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application [None req-3e45a90d-8de9-4fd1-b62f-f05f9a246161 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 23 14:59:16.524866 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 14:59:16.524866 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 14:59:16.524866 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 14:59:16.524866 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 14:59:16.524866 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 14:59:16.524866 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 14:59:16.524866 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 14:59:16.524866 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 14:59:16.524866 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 14:59:16.524866 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 14:59:16.524866 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 14:59:16.524866 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 14:59:16.524866 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 14:59:16.524866 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 14:59:16.524866 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 14:59:16.525775 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 14:59:16.525775 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 14:59:16.525775 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 14:59:16.525775 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 14:59:16.525775 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 14:59:16.525775 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 14:59:16.525775 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 14:59:16.525775 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 14:59:16.525775 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 14:59:16.525775 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 14:59:16.525775 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 14:59:16.525775 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application with Lock( Sep 23 14:59:16.525775 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 14:59:16.525775 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._enter() Sep 23 14:59:16.525775 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 14:59:16.525775 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 14:59:16.526562 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 14:59:16.526562 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.creator() Sep 23 14:59:16.526562 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 14:59:16.526562 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application created_value = creator( Sep 23 14:59:16.526562 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 14:59:16.526562 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 14:59:16.526562 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 14:59:16.526562 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 14:59:16.526562 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 14:59:16.526562 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 14:59:16.526562 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 23 14:59:16.526562 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Sep 23 14:59:16.527664 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 264/545] 10.222.0.55 () {56 vars in 1102 bytes} [Fri Sep 23 14:59:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 14:59:16.533699 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e45a90d-8de9-4fd1-b62f-f05f9a246161 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:16.587574 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:16.625261 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:16.652747 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 282/546] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Sep 23 14:59:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 14:59:16.667439 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e45a90d-8de9-4fd1-b62f-f05f9a246161 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:16.704352 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538fdfc6-8ed0-4190-90f7-bf1a8bbe417e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:16.705322 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538fdfc6-8ed0-4190-90f7-bf1a8bbe417e None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:16.705795 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538fdfc6-8ed0-4190-90f7-bf1a8bbe417e None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:16.706283 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538fdfc6-8ed0-4190-90f7-bf1a8bbe417e None admin] PATH_INFO: `/v3/roles` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:16.707209 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538fdfc6-8ed0-4190-90f7-bf1a8bbe417e None admin] RBAC: Adding query filter params (name=service) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:59:16.707703 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538fdfc6-8ed0-4190-90f7-bf1a8bbe417e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:16.713429 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538fdfc6-8ed0-4190-90f7-bf1a8bbe417e None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:16.726433 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 265/547] 10.222.0.55 () {56 vars in 1103 bytes} [Fri Sep 23 14:59:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 14:59:16.733718 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0100b577-f848-4485-9800-2f3dd3deec1d None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:16.857813 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0100b577-f848-4485-9800-2f3dd3deec1d None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:16.894560 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b569baff-c9c3-4693-ab60-1fb86ff1cf79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:16.895198 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b569baff-c9c3-4693-ab60-1fb86ff1cf79 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:16.895480 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b569baff-c9c3-4693-ab60-1fb86ff1cf79 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:16.895754 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b569baff-c9c3-4693-ab60-1fb86ff1cf79 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:16.906896 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b569baff-c9c3-4693-ab60-1fb86ff1cf79 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:16.908703 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b569baff-c9c3-4693-ab60-1fb86ff1cf79 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:16.924474 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-b569baff-c9c3-4693-ab60-1fb86ff1cf79 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 14:59:16.924474 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 14:59:16.924474 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 14:59:16.924474 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 14:59:16.924474 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 14:59:16.924474 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 14:59:16.924474 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 14:59:16.924474 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 14:59:16.924474 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 14:59:16.924474 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 14:59:16.924474 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 14:59:16.924474 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 14:59:16.924474 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 14:59:16.924474 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 14:59:16.924474 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 14:59:16.924474 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 14:59:16.925422 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 14:59:16.925422 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 14:59:16.925422 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 14:59:16.925422 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 14:59:16.925422 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 14:59:16.925422 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 14:59:16.925422 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 14:59:16.925422 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Sep 23 14:59:16.925422 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 14:59:16.925422 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Sep 23 14:59:16.925422 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 14:59:16.925422 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 14:59:16.925422 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 14:59:16.925422 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Sep 23 14:59:16.925422 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 14:59:16.925422 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Sep 23 14:59:16.925422 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 14:59:16.927298 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 14:59:16.927298 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 14:59:16.927298 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Sep 23 14:59:16.927298 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 283/548] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Sep 23 14:59:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 14:59:16.932442 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538fdfc6-8ed0-4190-90f7-bf1a8bbe417e None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:17.060277 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538fdfc6-8ed0-4190-90f7-bf1a8bbe417e None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:17.091061 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b569baff-c9c3-4693-ab60-1fb86ff1cf79 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:17.108353 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:17.121427 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef0c16d-0dbd-49ee-a0e3-f56ac8ad0be9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:17.122045 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef0c16d-0dbd-49ee-a0e3-f56ac8ad0be9 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:17.122319 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef0c16d-0dbd-49ee-a0e3-f56ac8ad0be9 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:17.122584 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef0c16d-0dbd-49ee-a0e3-f56ac8ad0be9 None admin] PATH_INFO: `/v3/domains` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:17.123220 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef0c16d-0dbd-49ee-a0e3-f56ac8ad0be9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:59:17.123525 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef0c16d-0dbd-49ee-a0e3-f56ac8ad0be9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:17.124906 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef0c16d-0dbd-49ee-a0e3-f56ac8ad0be9 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:17.147421 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:17.155357 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 266/549] 10.222.0.55 () {56 vars in 1109 bytes} [Fri Sep 23 14:59:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 14:59:17.156291 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef0c16d-0dbd-49ee-a0e3-f56ac8ad0be9 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:17.264671 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b569baff-c9c3-4693-ab60-1fb86ff1cf79 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:17.332841 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182e78fb-db3a-40f4-afd2-2e1d5e690e5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:17.333474 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182e78fb-db3a-40f4-afd2-2e1d5e690e5b None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:17.333751 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182e78fb-db3a-40f4-afd2-2e1d5e690e5b None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:17.334021 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182e78fb-db3a-40f4-afd2-2e1d5e690e5b None admin] PATH_INFO: `/v3/roles/service` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:17.339274 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef0c16d-0dbd-49ee-a0e3-f56ac8ad0be9 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:17.358538 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182e78fb-db3a-40f4-afd2-2e1d5e690e5b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 14:59:17.366764 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-182e78fb-db3a-40f4-afd2-2e1d5e690e5b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 23 14:59:17.366764 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 14:59:17.366764 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 14:59:17.366764 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 14:59:17.366764 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 14:59:17.366764 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 14:59:17.366764 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 14:59:17.366764 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 14:59:17.366764 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 14:59:17.366764 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 14:59:17.366764 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 14:59:17.366764 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 14:59:17.366764 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 14:59:17.366764 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 14:59:17.366764 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 14:59:17.366764 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 14:59:17.367750 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 14:59:17.367750 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 14:59:17.367750 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 14:59:17.367750 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 14:59:17.367750 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 14:59:17.367750 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 14:59:17.367750 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 14:59:17.367750 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 14:59:17.367750 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 14:59:17.367750 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 14:59:17.367750 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 14:59:17.367750 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Sep 23 14:59:17.367750 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 14:59:17.367750 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Sep 23 14:59:17.367750 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 14:59:17.367750 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 14:59:17.369186 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 14:59:17.369186 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Sep 23 14:59:17.369186 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 14:59:17.369186 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Sep 23 14:59:17.369186 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 14:59:17.369186 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 14:59:17.369186 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 14:59:17.369186 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 14:59:17.369186 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 14:59:17.369186 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 14:59:17.369186 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 23 14:59:17.369186 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Sep 23 14:59:17.369186 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 284/550] 10.222.0.55 () {56 vars in 1102 bytes} [Fri Sep 23 14:59:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 23 14:59:17.376676 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182e78fb-db3a-40f4-afd2-2e1d5e690e5b None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:17.415602 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b518eb9-1ace-436d-8613-686b5b0fffb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:17.416439 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b518eb9-1ace-436d-8613-686b5b0fffb2 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:17.416729 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b518eb9-1ace-436d-8613-686b5b0fffb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:17.416997 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b518eb9-1ace-436d-8613-686b5b0fffb2 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:17.424987 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b518eb9-1ace-436d-8613-686b5b0fffb2 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:17.426451 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b518eb9-1ace-436d-8613-686b5b0fffb2 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:17.439743 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application [None req-5b518eb9-1ace-436d-8613-686b5b0fffb2 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 23 14:59:17.439743 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 14:59:17.439743 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 14:59:17.439743 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 14:59:17.439743 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 14:59:17.439743 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 14:59:17.439743 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 14:59:17.439743 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 14:59:17.439743 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 14:59:17.439743 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 14:59:17.439743 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 14:59:17.439743 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 14:59:17.439743 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 23 14:59:17.439743 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 23 14:59:17.439743 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 23 14:59:17.439743 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 23 14:59:17.440728 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 14:59:17.440728 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 14:59:17.440728 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 23 14:59:17.440728 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 14:59:17.440728 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 23 14:59:17.440728 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 14:59:17.440728 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 14:59:17.440728 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 14:59:17.440728 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 14:59:17.440728 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 14:59:17.440728 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 14:59:17.440728 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application with Lock( Sep 23 14:59:17.440728 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 14:59:17.440728 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._enter() Sep 23 14:59:17.440728 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 14:59:17.440728 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 14:59:17.440728 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 14:59:17.440728 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.creator() Sep 23 14:59:17.441663 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 14:59:17.441663 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application created_value = creator( Sep 23 14:59:17.441663 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 23 14:59:17.441663 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 23 14:59:17.441663 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 23 14:59:17.441663 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 23 14:59:17.441663 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 23 14:59:17.441663 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 23 14:59:17.441663 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 23 14:59:17.441663 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Sep 23 14:59:17.441663 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 267/551] 10.222.0.55 () {56 vars in 1096 bytes} [Fri Sep 23 14:59:17 2022] GET /identity/v3/users/swift => generated 83 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 14:59:17.449052 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b518eb9-1ace-436d-8613-686b5b0fffb2 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:17.531827 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:17.549383 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:17.557663 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182e78fb-db3a-40f4-afd2-2e1d5e690e5b None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:17.597564 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc4ec03-620c-4c41-8ec5-508015c84333 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:17.598448 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc4ec03-620c-4c41-8ec5-508015c84333 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:17.598959 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc4ec03-620c-4c41-8ec5-508015c84333 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:17.599236 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc4ec03-620c-4c41-8ec5-508015c84333 None admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:17.599921 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc4ec03-620c-4c41-8ec5-508015c84333 None admin] RBAC: Adding query filter params (name=service) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:59:17.600242 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc4ec03-620c-4c41-8ec5-508015c84333 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:17.601608 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc4ec03-620c-4c41-8ec5-508015c84333 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:17.603167 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b518eb9-1ace-436d-8613-686b5b0fffb2 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:17.618642 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 285/552] 10.222.0.55 () {56 vars in 1103 bytes} [Fri Sep 23 14:59:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:59:17.620282 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efba25c9-79ba-4f1f-928a-f3de562f9e27 None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:17.620576 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efba25c9-79ba-4f1f-928a-f3de562f9e27 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:17.620855 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efba25c9-79ba-4f1f-928a-f3de562f9e27 None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:17.626580 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 286/553] 10.222.0.55 () {54 vars in 927 bytes} [Fri Sep 23 14:59:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 14:59:17.627419 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efba25c9-79ba-4f1f-928a-f3de562f9e27 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:17.646690 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcacb80b-36dd-434a-80c6-6897af321e6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:17.648084 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcacb80b-36dd-434a-80c6-6897af321e6d None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:17.648640 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcacb80b-36dd-434a-80c6-6897af321e6d None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:17.649172 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcacb80b-36dd-434a-80c6-6897af321e6d None admin] PATH_INFO: `/v3/users` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:17.650287 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcacb80b-36dd-434a-80c6-6897af321e6d None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:59:17.650811 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcacb80b-36dd-434a-80c6-6897af321e6d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:17.652522 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcacb80b-36dd-434a-80c6-6897af321e6d None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:17.673755 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 268/554] 10.222.0.55 () {56 vars in 1135 bytes} [Fri Sep 23 14:59:17 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 14:59:17.675569 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e50fd38-417d-4b7c-94dd-6b4942989c30 None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:17.676106 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e50fd38-417d-4b7c-94dd-6b4942989c30 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:17.676586 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e50fd38-417d-4b7c-94dd-6b4942989c30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:17.700968 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-1e50fd38-417d-4b7c-94dd-6b4942989c30 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 14:59:17.703223 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-1e50fd38-417d-4b7c-94dd-6b4942989c30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 14:59:17.765028 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efba25c9-79ba-4f1f-928a-f3de562f9e27 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:17.800638 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 269/555] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Sep 23 14:59:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 14:59:17.804628 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e50fd38-417d-4b7c-94dd-6b4942989c30 None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:17.818528 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573013af-7f86-4380-b1c8-593573cd86c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:17.820506 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573013af-7f86-4380-b1c8-593573cd86c2 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:17.820788 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573013af-7f86-4380-b1c8-593573cd86c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:17.821060 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573013af-7f86-4380-b1c8-593573cd86c2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:17.846007 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573013af-7f86-4380-b1c8-593573cd86c2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:17.847835 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573013af-7f86-4380-b1c8-593573cd86c2 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:17.863533 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:17.881607 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-573013af-7f86-4380-b1c8-593573cd86c2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 14:59:17.881607 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 14:59:17.881607 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 14:59:17.881607 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 14:59:17.881607 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 14:59:17.881607 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 14:59:17.881607 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 14:59:17.881607 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 14:59:17.881607 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 14:59:17.881607 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 14:59:17.881607 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 14:59:17.881607 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 14:59:17.881607 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 14:59:17.881607 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 14:59:17.881607 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 14:59:17.881607 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 14:59:17.882685 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 14:59:17.882685 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 14:59:17.882685 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 14:59:17.882685 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 14:59:17.882685 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 14:59:17.882685 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 14:59:17.882685 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 14:59:17.882685 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Sep 23 14:59:17.882685 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 14:59:17.882685 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Sep 23 14:59:17.882685 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 14:59:17.882685 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 14:59:17.882685 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 14:59:17.882685 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Sep 23 14:59:17.882685 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 14:59:17.882685 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Sep 23 14:59:17.882685 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 14:59:17.883595 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 14:59:17.883595 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 14:59:17.883595 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Sep 23 14:59:17.883595 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 287/556] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Sep 23 14:59:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 14:59:17.889881 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573013af-7f86-4380-b1c8-593573cd86c2 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:17.910449 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:17.994329 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e50fd38-417d-4b7c-94dd-6b4942989c30 None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:18.044980 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43794122-9083-4e40-a8eb-0fd242f8dbbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:18.045641 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43794122-9083-4e40-a8eb-0fd242f8dbbe None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:18.045919 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43794122-9083-4e40-a8eb-0fd242f8dbbe None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:18.046190 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43794122-9083-4e40-a8eb-0fd242f8dbbe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:18.094443 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573013af-7f86-4380-b1c8-593573cd86c2 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:18.130543 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43794122-9083-4e40-a8eb-0fd242f8dbbe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:18.132390 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43794122-9083-4e40-a8eb-0fd242f8dbbe None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:18.141164 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf7633e-02bb-4c08-b9f2-58184403a236 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:18.141789 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf7633e-02bb-4c08-b9f2-58184403a236 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:18.142072 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf7633e-02bb-4c08-b9f2-58184403a236 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:18.142347 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf7633e-02bb-4c08-b9f2-58184403a236 None admin] PATH_INFO: `/v3/domains` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:18.142999 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf7633e-02bb-4c08-b9f2-58184403a236 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:59:18.143351 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf7633e-02bb-4c08-b9f2-58184403a236 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:18.145087 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf7633e-02bb-4c08-b9f2-58184403a236 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:18.158356 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 288/557] 10.222.0.55 () {56 vars in 1109 bytes} [Fri Sep 23 14:59:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:59:18.160757 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a03015-4921-4321-b6a4-02e33cac5379 None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:18.161048 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a03015-4921-4321-b6a4-02e33cac5379 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:18.161324 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a03015-4921-4321-b6a4-02e33cac5379 None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:18.162733 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 289/558] 10.222.0.55 () {54 vars in 927 bytes} [Fri Sep 23 14:59:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 14:59:18.163409 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a03015-4921-4321-b6a4-02e33cac5379 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:18.277868 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:18.315297 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:18.353810 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a03015-4921-4321-b6a4-02e33cac5379 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:18.360683 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 270/559] 10.222.0.55 () {58 vars in 1282 bytes} [Fri Sep 23 14:59:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 560 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 23 14:59:18.361685 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43794122-9083-4e40-a8eb-0fd242f8dbbe None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:18.396269 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d4bf24-3fb2-4356-962d-5f3699198d03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:18.396920 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d4bf24-3fb2-4356-962d-5f3699198d03 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:18.397204 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d4bf24-3fb2-4356-962d-5f3699198d03 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:18.397480 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d4bf24-3fb2-4356-962d-5f3699198d03 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:18.409713 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d4bf24-3fb2-4356-962d-5f3699198d03 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 14:59:18.414706 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-97d4bf24-3fb2-4356-962d-5f3699198d03 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 23 14:59:18.414706 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 14:59:18.414706 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 14:59:18.414706 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 14:59:18.414706 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 14:59:18.414706 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 14:59:18.414706 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 14:59:18.414706 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 14:59:18.414706 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 14:59:18.414706 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 14:59:18.414706 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 14:59:18.414706 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 14:59:18.414706 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 14:59:18.414706 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 14:59:18.414706 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 14:59:18.414706 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 14:59:18.415835 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 14:59:18.415835 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 14:59:18.415835 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 14:59:18.415835 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 14:59:18.415835 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 14:59:18.415835 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 14:59:18.415835 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 14:59:18.415835 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 14:59:18.415835 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 14:59:18.415835 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 14:59:18.415835 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 14:59:18.415835 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Sep 23 14:59:18.415835 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 14:59:18.415835 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Sep 23 14:59:18.415835 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 14:59:18.415835 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 14:59:18.416713 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 14:59:18.416713 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Sep 23 14:59:18.416713 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 14:59:18.416713 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Sep 23 14:59:18.416713 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 14:59:18.416713 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 14:59:18.416713 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 14:59:18.416713 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 14:59:18.416713 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 14:59:18.416713 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 14:59:18.416713 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 23 14:59:18.416713 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Sep 23 14:59:18.416713 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 290/560] 10.222.0.55 () {56 vars in 1102 bytes} [Fri Sep 23 14:59:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 14:59:18.417932 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fb5afd-2f4f-4320-80fb-dbb566703d0e None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:18.417932 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fb5afd-2f4f-4320-80fb-dbb566703d0e None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:18.417932 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fb5afd-2f4f-4320-80fb-dbb566703d0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:18.435623 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-95fb5afd-2f4f-4320-80fb-dbb566703d0e None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 14:59:18.436859 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-95fb5afd-2f4f-4320-80fb-dbb566703d0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 14:59:18.471558 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43794122-9083-4e40-a8eb-0fd242f8dbbe None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:18.508662 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12eebf2f-0371-4183-bf26-04a1de3d0688 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:18.510154 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12eebf2f-0371-4183-bf26-04a1de3d0688 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:18.510431 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12eebf2f-0371-4183-bf26-04a1de3d0688 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:18.510663 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12eebf2f-0371-4183-bf26-04a1de3d0688 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:18.512101 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 291/561] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Sep 23 14:59:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 14:59:18.513237 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fb5afd-2f4f-4320-80fb-dbb566703d0e None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:18.517794 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:18.522839 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12eebf2f-0371-4183-bf26-04a1de3d0688 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:18.524552 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12eebf2f-0371-4183-bf26-04a1de3d0688 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:18.537395 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application [None req-12eebf2f-0371-4183-bf26-04a1de3d0688 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 23 14:59:18.537395 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 14:59:18.537395 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 14:59:18.537395 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 14:59:18.537395 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 14:59:18.537395 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 14:59:18.537395 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 14:59:18.537395 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 14:59:18.537395 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 14:59:18.537395 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 14:59:18.537395 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 14:59:18.537395 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 14:59:18.537395 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 23 14:59:18.537395 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 23 14:59:18.537395 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 23 14:59:18.537395 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 23 14:59:18.538632 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 14:59:18.538632 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 14:59:18.538632 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 23 14:59:18.538632 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 14:59:18.538632 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 23 14:59:18.538632 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 14:59:18.538632 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 14:59:18.538632 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 14:59:18.538632 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 14:59:18.538632 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 14:59:18.538632 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 14:59:18.538632 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application with Lock( Sep 23 14:59:18.538632 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 14:59:18.538632 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._enter() Sep 23 14:59:18.538632 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 14:59:18.538632 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 14:59:18.538632 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 14:59:18.538632 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.creator() Sep 23 14:59:18.539720 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 14:59:18.539720 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application created_value = creator( Sep 23 14:59:18.539720 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 23 14:59:18.539720 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 23 14:59:18.539720 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 23 14:59:18.539720 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 23 14:59:18.539720 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 23 14:59:18.539720 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 23 14:59:18.539720 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 23 14:59:18.539720 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Sep 23 14:59:18.541860 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 271/562] 10.222.0.55 () {56 vars in 1093 bytes} [Fri Sep 23 14:59:18 2022] GET /identity/v3/users/nova => generated 82 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 14:59:18.542849 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12eebf2f-0371-4183-bf26-04a1de3d0688 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:18.608404 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:18.655255 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fb5afd-2f4f-4320-80fb-dbb566703d0e None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:18.658410 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12eebf2f-0371-4183-bf26-04a1de3d0688 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:18.692400 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c72d7fc-c541-4a28-bdca-cf3e8ee3d94a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:18.693030 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c72d7fc-c541-4a28-bdca-cf3e8ee3d94a None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:18.693306 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c72d7fc-c541-4a28-bdca-cf3e8ee3d94a None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:18.693577 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c72d7fc-c541-4a28-bdca-cf3e8ee3d94a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:18.695450 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de67dee-3741-4e09-a8f5-b806df77c254 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:18.696530 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de67dee-3741-4e09-a8f5-b806df77c254 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:18.697946 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de67dee-3741-4e09-a8f5-b806df77c254 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:18.697946 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de67dee-3741-4e09-a8f5-b806df77c254 None admin] PATH_INFO: `/v3/roles` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:18.698405 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de67dee-3741-4e09-a8f5-b806df77c254 None admin] RBAC: Adding query filter params (name=service) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:59:18.698720 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de67dee-3741-4e09-a8f5-b806df77c254 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:18.700194 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de67dee-3741-4e09-a8f5-b806df77c254 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:18.705621 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c72d7fc-c541-4a28-bdca-cf3e8ee3d94a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:18.707089 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c72d7fc-c541-4a28-bdca-cf3e8ee3d94a None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:18.708688 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 272/563] 10.222.0.55 () {56 vars in 1103 bytes} [Fri Sep 23 14:59:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:59:18.709496 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de67dee-3741-4e09-a8f5-b806df77c254 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:18.729905 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-5c72d7fc-c541-4a28-bdca-cf3e8ee3d94a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 14:59:18.729905 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 14:59:18.729905 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 14:59:18.729905 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 14:59:18.729905 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 14:59:18.729905 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 14:59:18.729905 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 14:59:18.729905 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 14:59:18.729905 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 14:59:18.729905 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 14:59:18.729905 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 14:59:18.729905 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 14:59:18.729905 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 14:59:18.729905 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 14:59:18.729905 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 14:59:18.729905 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 14:59:18.733061 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 14:59:18.733061 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 14:59:18.733061 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 14:59:18.733061 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 14:59:18.733061 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 14:59:18.733061 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 14:59:18.733061 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 14:59:18.733061 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Sep 23 14:59:18.733061 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 14:59:18.733061 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Sep 23 14:59:18.733061 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 14:59:18.733061 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 14:59:18.733061 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 14:59:18.733061 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Sep 23 14:59:18.733061 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 14:59:18.733061 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Sep 23 14:59:18.733061 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 14:59:18.733958 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 14:59:18.733958 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 14:59:18.733958 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Sep 23 14:59:18.733958 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 292/564] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Sep 23 14:59:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 14:59:18.733958 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f564c3-b7e3-488e-9024-649c4f748209 None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:18.733958 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f564c3-b7e3-488e-9024-649c4f748209 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:18.733958 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f564c3-b7e3-488e-9024-649c4f748209 None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:18.736877 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 293/565] 10.222.0.55 () {54 vars in 927 bytes} [Fri Sep 23 14:59:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 14:59:18.736877 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f564c3-b7e3-488e-9024-649c4f748209 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:18.788992 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:18.834074 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de67dee-3741-4e09-a8f5-b806df77c254 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:18.845191 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f564c3-b7e3-488e-9024-649c4f748209 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:18.857380 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:18.868684 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f3cbce-0c85-4f32-8742-9ed94248f242 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:18.870844 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f3cbce-0c85-4f32-8742-9ed94248f242 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:18.871137 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f3cbce-0c85-4f32-8742-9ed94248f242 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:18.871415 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f3cbce-0c85-4f32-8742-9ed94248f242 None admin] PATH_INFO: `/v3/users` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:18.872175 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f3cbce-0c85-4f32-8742-9ed94248f242 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:59:18.872485 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f3cbce-0c85-4f32-8742-9ed94248f242 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:18.873921 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f3cbce-0c85-4f32-8742-9ed94248f242 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:18.880968 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3939d48-84b7-46dc-822f-164a0e4f38aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:18.881600 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3939d48-84b7-46dc-822f-164a0e4f38aa None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:18.881949 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3939d48-84b7-46dc-822f-164a0e4f38aa None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:18.882156 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3939d48-84b7-46dc-822f-164a0e4f38aa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:18.891845 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 273/566] 10.222.0.55 () {56 vars in 1133 bytes} [Fri Sep 23 14:59:18 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:59:18.892689 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f3cbce-0c85-4f32-8742-9ed94248f242 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:18.897883 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3939d48-84b7-46dc-822f-164a0e4f38aa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:18.899378 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3939d48-84b7-46dc-822f-164a0e4f38aa None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:18.914409 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-f3939d48-84b7-46dc-822f-164a0e4f38aa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 14:59:18.914409 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 14:59:18.914409 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 14:59:18.914409 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 14:59:18.914409 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 14:59:18.914409 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 14:59:18.914409 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 14:59:18.914409 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 14:59:18.914409 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 14:59:18.914409 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 14:59:18.914409 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 14:59:18.914409 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 14:59:18.914409 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 14:59:18.914409 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 14:59:18.914409 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 14:59:18.914409 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 14:59:18.915501 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 14:59:18.915501 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 14:59:18.915501 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 14:59:18.915501 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 14:59:18.915501 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 14:59:18.915501 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 14:59:18.915501 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 14:59:18.915501 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Sep 23 14:59:18.915501 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 14:59:18.915501 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Sep 23 14:59:18.915501 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 14:59:18.915501 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 14:59:18.915501 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 14:59:18.915501 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Sep 23 14:59:18.915501 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 14:59:18.915501 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Sep 23 14:59:18.915501 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 14:59:18.916429 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 14:59:18.916429 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 14:59:18.916429 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Sep 23 14:59:18.916429 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 294/567] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Sep 23 14:59:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 14:59:18.916673 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84198deb-667b-4116-bbdf-ba99a7cb3614 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:18.916899 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84198deb-667b-4116-bbdf-ba99a7cb3614 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:18.917306 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84198deb-667b-4116-bbdf-ba99a7cb3614 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:18.934758 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-84198deb-667b-4116-bbdf-ba99a7cb3614 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 14:59:18.936083 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-84198deb-667b-4116-bbdf-ba99a7cb3614 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 14:59:18.993953 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f3cbce-0c85-4f32-8742-9ed94248f242 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:19.003197 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 295/568] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Sep 23 14:59:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 14:59:19.004397 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84198deb-667b-4116-bbdf-ba99a7cb3614 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:19.012381 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:19.032889 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e109803-3632-4397-8d08-aa8bff10ead7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:19.033592 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e109803-3632-4397-8d08-aa8bff10ead7 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:19.033880 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e109803-3632-4397-8d08-aa8bff10ead7 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:19.034160 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e109803-3632-4397-8d08-aa8bff10ead7 None admin] PATH_INFO: `/v3/domains` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:19.034807 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e109803-3632-4397-8d08-aa8bff10ead7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:59:19.035113 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e109803-3632-4397-8d08-aa8bff10ead7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:19.036765 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e109803-3632-4397-8d08-aa8bff10ead7 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:19.051090 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 274/569] 10.222.0.55 () {56 vars in 1109 bytes} [Fri Sep 23 14:59:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:59:19.052388 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e109803-3632-4397-8d08-aa8bff10ead7 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:19.115210 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:19.125409 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84198deb-667b-4116-bbdf-ba99a7cb3614 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:19.160930 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef414457-44a6-41da-a154-8007ff084246 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:19.161917 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef414457-44a6-41da-a154-8007ff084246 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:19.162368 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef414457-44a6-41da-a154-8007ff084246 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:19.162752 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef414457-44a6-41da-a154-8007ff084246 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:19.166125 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e109803-3632-4397-8d08-aa8bff10ead7 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:19.175052 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef414457-44a6-41da-a154-8007ff084246 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 14:59:19.179840 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-ef414457-44a6-41da-a154-8007ff084246 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 23 14:59:19.179840 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 14:59:19.179840 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 14:59:19.179840 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 14:59:19.179840 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 14:59:19.179840 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 14:59:19.179840 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 14:59:19.179840 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 14:59:19.179840 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 14:59:19.179840 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 14:59:19.179840 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 14:59:19.179840 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 14:59:19.179840 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 14:59:19.179840 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 14:59:19.179840 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 14:59:19.179840 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 14:59:19.181734 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 14:59:19.181734 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 14:59:19.181734 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 14:59:19.181734 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 14:59:19.181734 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 14:59:19.181734 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 14:59:19.181734 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 14:59:19.181734 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 14:59:19.181734 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 14:59:19.181734 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 14:59:19.181734 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 14:59:19.181734 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Sep 23 14:59:19.181734 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 14:59:19.181734 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Sep 23 14:59:19.181734 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 14:59:19.181734 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 14:59:19.182850 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 14:59:19.182850 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Sep 23 14:59:19.182850 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 14:59:19.182850 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Sep 23 14:59:19.182850 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 14:59:19.182850 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 14:59:19.182850 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 14:59:19.182850 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 14:59:19.182850 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 14:59:19.182850 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 14:59:19.182850 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 23 14:59:19.182850 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Sep 23 14:59:19.182850 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 296/570] 10.222.0.55 () {56 vars in 1102 bytes} [Fri Sep 23 14:59:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 14:59:19.182850 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef414457-44a6-41da-a154-8007ff084246 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:19.208128 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a6d5d3-ab65-4536-8778-27ff75b211dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:19.208767 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a6d5d3-ab65-4536-8778-27ff75b211dd None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:19.209046 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a6d5d3-ab65-4536-8778-27ff75b211dd None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:19.209318 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a6d5d3-ab65-4536-8778-27ff75b211dd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:19.221175 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a6d5d3-ab65-4536-8778-27ff75b211dd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:19.222640 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a6d5d3-ab65-4536-8778-27ff75b211dd None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:19.237626 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application [None req-49a6d5d3-ab65-4536-8778-27ff75b211dd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 14:59:19.237626 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 14:59:19.237626 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 14:59:19.237626 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 14:59:19.237626 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 14:59:19.237626 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 14:59:19.237626 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 14:59:19.237626 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 14:59:19.237626 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 14:59:19.237626 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 14:59:19.237626 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 14:59:19.237626 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 14:59:19.237626 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 14:59:19.237626 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 14:59:19.237626 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 14:59:19.237626 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 14:59:19.238616 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 14:59:19.238616 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 14:59:19.238616 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 14:59:19.238616 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 14:59:19.238616 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 14:59:19.238616 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 14:59:19.238616 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 14:59:19.238616 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application with Lock( Sep 23 14:59:19.238616 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 14:59:19.238616 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._enter() Sep 23 14:59:19.238616 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 14:59:19.238616 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 14:59:19.238616 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 14:59:19.238616 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.creator() Sep 23 14:59:19.238616 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 14:59:19.238616 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application created_value = creator( Sep 23 14:59:19.238616 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 14:59:19.239515 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 14:59:19.239515 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 14:59:19.239515 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Sep 23 14:59:19.245100 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 275/571] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Sep 23 14:59:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 14:59:19.245749 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a6d5d3-ab65-4536-8778-27ff75b211dd None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:19.275306 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:19.292406 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef414457-44a6-41da-a154-8007ff084246 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:19.326636 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade1d234-62e7-4327-afec-7add27c708d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:19.327244 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade1d234-62e7-4327-afec-7add27c708d1 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:19.327525 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade1d234-62e7-4327-afec-7add27c708d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:19.327790 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade1d234-62e7-4327-afec-7add27c708d1 None admin] PATH_INFO: `/v3/domains` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:19.328456 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade1d234-62e7-4327-afec-7add27c708d1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:59:19.328763 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade1d234-62e7-4327-afec-7add27c708d1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:19.330138 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade1d234-62e7-4327-afec-7add27c708d1 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:19.342256 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 297/572] 10.222.0.55 () {56 vars in 1109 bytes} [Fri Sep 23 14:59:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:59:19.342895 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade1d234-62e7-4327-afec-7add27c708d1 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:19.354223 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a6d5d3-ab65-4536-8778-27ff75b211dd None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:19.358111 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:19.389260 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6099b3-f7e2-4c7a-a45d-3929a97730e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:19.389889 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6099b3-f7e2-4c7a-a45d-3929a97730e0 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:19.390257 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6099b3-f7e2-4c7a-a45d-3929a97730e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:19.390536 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6099b3-f7e2-4c7a-a45d-3929a97730e0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:19.397125 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6099b3-f7e2-4c7a-a45d-3929a97730e0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:19.398579 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6099b3-f7e2-4c7a-a45d-3929a97730e0 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:19.410425 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application [None req-1d6099b3-f7e2-4c7a-a45d-3929a97730e0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 14:59:19.410425 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 14:59:19.410425 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 14:59:19.410425 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 14:59:19.410425 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 14:59:19.410425 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 14:59:19.410425 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 14:59:19.410425 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 14:59:19.410425 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 14:59:19.410425 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 14:59:19.410425 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 14:59:19.410425 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 14:59:19.410425 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 23 14:59:19.410425 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 23 14:59:19.410425 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 23 14:59:19.411405 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 23 14:59:19.411405 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 14:59:19.411405 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 14:59:19.411405 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 14:59:19.411405 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 14:59:19.411405 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 14:59:19.411405 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 14:59:19.411405 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 14:59:19.411405 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application with Lock( Sep 23 14:59:19.411405 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 14:59:19.411405 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._enter() Sep 23 14:59:19.411405 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 14:59:19.411405 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 14:59:19.411405 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 14:59:19.411405 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.creator() Sep 23 14:59:19.411405 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 14:59:19.411405 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application created_value = creator( Sep 23 14:59:19.412336 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 23 14:59:19.412336 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 23 14:59:19.412336 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 23 14:59:19.412336 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 23 14:59:19.412336 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 23 14:59:19.412336 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 23 14:59:19.412336 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 14:59:19.412336 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Sep 23 14:59:19.412336 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 276/573] 10.222.0.55 () {56 vars in 1111 bytes} [Fri Sep 23 14:59:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 14:59:19.412844 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6099b3-f7e2-4c7a-a45d-3929a97730e0 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:19.451422 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade1d234-62e7-4327-afec-7add27c708d1 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:19.487916 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2cda0a-cb7e-4885-89f1-095f791f47d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:19.488565 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2cda0a-cb7e-4885-89f1-095f791f47d0 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:19.488845 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2cda0a-cb7e-4885-89f1-095f791f47d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:19.489117 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2cda0a-cb7e-4885-89f1-095f791f47d0 None admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:19.489820 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2cda0a-cb7e-4885-89f1-095f791f47d0 None admin] RBAC: Adding query filter params (name=service) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:59:19.490105 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2cda0a-cb7e-4885-89f1-095f791f47d0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:19.491468 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2cda0a-cb7e-4885-89f1-095f791f47d0 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:19.504418 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 298/574] 10.222.0.55 () {56 vars in 1103 bytes} [Fri Sep 23 14:59:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:59:19.504418 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2cda0a-cb7e-4885-89f1-095f791f47d0 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:19.505394 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:19.518097 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6099b3-f7e2-4c7a-a45d-3929a97730e0 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:19.551651 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a98540-f833-4d9e-ad77-11db0a7453ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:19.552294 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a98540-f833-4d9e-ad77-11db0a7453ac None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:19.552583 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a98540-f833-4d9e-ad77-11db0a7453ac None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:19.552857 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a98540-f833-4d9e-ad77-11db0a7453ac None admin] PATH_INFO: `/v3/domains` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:19.553493 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a98540-f833-4d9e-ad77-11db0a7453ac None admin] RBAC: Adding query filter params (name=Default) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:59:19.553822 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a98540-f833-4d9e-ad77-11db0a7453ac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:19.555190 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a98540-f833-4d9e-ad77-11db0a7453ac None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:19.567491 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 277/575] 10.222.0.55 () {56 vars in 1109 bytes} [Fri Sep 23 14:59:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:59:19.568159 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a98540-f833-4d9e-ad77-11db0a7453ac None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:19.582830 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:19.608046 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2cda0a-cb7e-4885-89f1-095f791f47d0 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:19.643931 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44919575-481c-4f7f-8d73-59a33bb4cf4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:19.644613 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44919575-481c-4f7f-8d73-59a33bb4cf4f None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:19.644897 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44919575-481c-4f7f-8d73-59a33bb4cf4f None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:19.645174 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44919575-481c-4f7f-8d73-59a33bb4cf4f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:19.659555 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44919575-481c-4f7f-8d73-59a33bb4cf4f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 14:59:19.664814 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-44919575-481c-4f7f-8d73-59a33bb4cf4f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 23 14:59:19.664814 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 14:59:19.664814 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 14:59:19.664814 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 14:59:19.664814 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 14:59:19.664814 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 14:59:19.664814 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 14:59:19.664814 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 14:59:19.664814 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 14:59:19.664814 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 14:59:19.664814 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 14:59:19.664814 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 14:59:19.664814 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 14:59:19.664814 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 14:59:19.664814 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 14:59:19.664814 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 14:59:19.666388 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 14:59:19.666388 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 14:59:19.666388 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 14:59:19.666388 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 14:59:19.666388 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 14:59:19.666388 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 14:59:19.666388 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 14:59:19.666388 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 14:59:19.666388 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 14:59:19.666388 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 14:59:19.666388 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 14:59:19.666388 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Sep 23 14:59:19.666388 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 14:59:19.666388 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Sep 23 14:59:19.666388 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 14:59:19.666388 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 14:59:19.667590 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 14:59:19.667590 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Sep 23 14:59:19.667590 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 14:59:19.667590 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Sep 23 14:59:19.667590 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 14:59:19.667590 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 14:59:19.667590 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 14:59:19.667590 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 14:59:19.667590 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 14:59:19.667590 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 14:59:19.667590 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 23 14:59:19.667590 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Sep 23 14:59:19.667590 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 299/576] 10.222.0.55 () {56 vars in 1102 bytes} [Fri Sep 23 14:59:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 14:59:19.667590 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44919575-481c-4f7f-8d73-59a33bb4cf4f None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:19.675794 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a98540-f833-4d9e-ad77-11db0a7453ac None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:19.710220 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e461ebdc-89d4-4188-88f9-43678e16fc32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:19.710771 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e461ebdc-89d4-4188-88f9-43678e16fc32 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:19.711049 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e461ebdc-89d4-4188-88f9-43678e16fc32 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:19.711319 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e461ebdc-89d4-4188-88f9-43678e16fc32 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:19.718763 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e461ebdc-89d4-4188-88f9-43678e16fc32 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:19.720322 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e461ebdc-89d4-4188-88f9-43678e16fc32 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:19.732409 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application [None req-e461ebdc-89d4-4188-88f9-43678e16fc32 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 23 14:59:19.732409 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 14:59:19.732409 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 14:59:19.732409 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 14:59:19.732409 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 14:59:19.732409 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 14:59:19.732409 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 14:59:19.732409 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 14:59:19.732409 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 14:59:19.732409 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 14:59:19.732409 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 14:59:19.732409 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 14:59:19.732409 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 23 14:59:19.732409 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 23 14:59:19.732409 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 23 14:59:19.732409 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 23 14:59:19.733425 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 14:59:19.733425 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 14:59:19.733425 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 23 14:59:19.733425 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 14:59:19.733425 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 23 14:59:19.733425 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 14:59:19.733425 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 14:59:19.733425 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 14:59:19.733425 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 14:59:19.733425 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 14:59:19.733425 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 14:59:19.733425 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application with Lock( Sep 23 14:59:19.733425 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 14:59:19.733425 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._enter() Sep 23 14:59:19.733425 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 14:59:19.733425 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 14:59:19.733425 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 14:59:19.733425 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.creator() Sep 23 14:59:19.734747 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 14:59:19.734747 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application created_value = creator( Sep 23 14:59:19.734747 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 23 14:59:19.734747 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 23 14:59:19.734747 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 23 14:59:19.734747 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 23 14:59:19.734747 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 23 14:59:19.734747 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 23 14:59:19.734747 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 23 14:59:19.734747 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Sep 23 14:59:19.734747 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 278/577] 10.222.0.55 () {56 vars in 1099 bytes} [Fri Sep 23 14:59:19 2022] GET /identity/v3/users/glance => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 14:59:19.734747 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e461ebdc-89d4-4188-88f9-43678e16fc32 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:19.745894 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:19.776493 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44919575-481c-4f7f-8d73-59a33bb4cf4f None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:19.811028 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c5989e-453b-49e8-98f9-4a26f0d36cac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:19.811632 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c5989e-453b-49e8-98f9-4a26f0d36cac None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:19.811911 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c5989e-453b-49e8-98f9-4a26f0d36cac None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:19.812208 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c5989e-453b-49e8-98f9-4a26f0d36cac None admin] PATH_INFO: `/v3/projects` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:19.812900 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c5989e-453b-49e8-98f9-4a26f0d36cac None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:59:19.813201 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c5989e-453b-49e8-98f9-4a26f0d36cac None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:19.814622 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c5989e-453b-49e8-98f9-4a26f0d36cac None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:19.816221 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:19.827072 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 300/578] 10.222.0.55 () {56 vars in 1148 bytes} [Fri Sep 23 14:59:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:59:19.828800 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c5989e-453b-49e8-98f9-4a26f0d36cac None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:19.837732 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e461ebdc-89d4-4188-88f9-43678e16fc32 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:19.873816 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c100f42-5650-47dd-9275-f025307f1436 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:19.874584 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c100f42-5650-47dd-9275-f025307f1436 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:19.874973 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c100f42-5650-47dd-9275-f025307f1436 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:19.875388 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c100f42-5650-47dd-9275-f025307f1436 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:19.886446 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c100f42-5650-47dd-9275-f025307f1436 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:19.887896 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c100f42-5650-47dd-9275-f025307f1436 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:19.901914 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application [None req-4c100f42-5650-47dd-9275-f025307f1436 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 14:59:19.901914 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 14:59:19.901914 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 14:59:19.901914 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 14:59:19.901914 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 14:59:19.901914 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 14:59:19.901914 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 14:59:19.901914 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 14:59:19.901914 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 14:59:19.901914 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 14:59:19.901914 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 14:59:19.901914 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 14:59:19.901914 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 14:59:19.901914 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 14:59:19.901914 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 14:59:19.901914 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 14:59:19.907771 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 14:59:19.907771 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 14:59:19.907771 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 14:59:19.907771 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 14:59:19.907771 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 14:59:19.907771 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 14:59:19.907771 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 14:59:19.907771 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application with Lock( Sep 23 14:59:19.907771 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 14:59:19.907771 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._enter() Sep 23 14:59:19.907771 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 14:59:19.907771 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 14:59:19.907771 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 14:59:19.907771 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.creator() Sep 23 14:59:19.907771 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 14:59:19.907771 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application created_value = creator( Sep 23 14:59:19.907771 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 14:59:19.908911 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 14:59:19.908911 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 14:59:19.908911 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Sep 23 14:59:19.908911 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 279/579] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Sep 23 14:59:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 14:59:19.908911 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c100f42-5650-47dd-9275-f025307f1436 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:19.929324 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c5989e-453b-49e8-98f9-4a26f0d36cac None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:19.963484 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1475828-88cb-4e1d-9b0f-36f164d60efe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:19.964050 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1475828-88cb-4e1d-9b0f-36f164d60efe None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:19.964391 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1475828-88cb-4e1d-9b0f-36f164d60efe None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:19.964687 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1475828-88cb-4e1d-9b0f-36f164d60efe None admin] PATH_INFO: `/v3/projects/service` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:19.968228 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:19.971436 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1475828-88cb-4e1d-9b0f-36f164d60efe None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:19.972894 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1475828-88cb-4e1d-9b0f-36f164d60efe None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:19.985704 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-c1475828-88cb-4e1d-9b0f-36f164d60efe None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 14:59:19.985704 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 14:59:19.985704 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 14:59:19.985704 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 14:59:19.985704 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 14:59:19.985704 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 14:59:19.985704 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 14:59:19.985704 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 14:59:19.985704 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 14:59:19.985704 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 14:59:19.985704 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 14:59:19.985704 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 14:59:19.985704 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 23 14:59:19.985704 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 23 14:59:19.985704 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 23 14:59:19.991537 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 23 14:59:19.991537 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 14:59:19.991537 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 14:59:19.991537 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 14:59:19.991537 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 14:59:19.991537 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 14:59:19.991537 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 14:59:19.991537 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 14:59:19.991537 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Sep 23 14:59:19.991537 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 14:59:19.991537 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Sep 23 14:59:19.991537 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 14:59:19.991537 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 14:59:19.991537 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 14:59:19.991537 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Sep 23 14:59:19.991537 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 14:59:19.991537 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Sep 23 14:59:19.992691 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 23 14:59:19.992691 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 23 14:59:19.992691 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 23 14:59:19.992691 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 23 14:59:19.992691 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 23 14:59:19.992691 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 23 14:59:19.992691 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 14:59:19.992691 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Sep 23 14:59:19.992691 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 301/580] 10.222.0.55 () {56 vars in 1111 bytes} [Fri Sep 23 14:59:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 14:59:19.992691 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1475828-88cb-4e1d-9b0f-36f164d60efe None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:20.012840 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c100f42-5650-47dd-9275-f025307f1436 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:20.045065 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:20.047842 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eda53d7-6929-4dcd-ba69-819a0d5cb3aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:20.048530 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eda53d7-6929-4dcd-ba69-819a0d5cb3aa None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:20.048795 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eda53d7-6929-4dcd-ba69-819a0d5cb3aa None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:20.049071 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eda53d7-6929-4dcd-ba69-819a0d5cb3aa None admin] PATH_INFO: `/v3/roles` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:20.049731 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eda53d7-6929-4dcd-ba69-819a0d5cb3aa None admin] RBAC: Adding query filter params (name=service) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:59:20.050131 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eda53d7-6929-4dcd-ba69-819a0d5cb3aa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:20.051501 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eda53d7-6929-4dcd-ba69-819a0d5cb3aa None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:20.060303 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 280/581] 10.222.0.55 () {56 vars in 1103 bytes} [Fri Sep 23 14:59:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:59:20.063893 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eda53d7-6929-4dcd-ba69-819a0d5cb3aa None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:20.090240 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1475828-88cb-4e1d-9b0f-36f164d60efe None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:20.127768 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f01d69-e716-42ba-9244-f8b0ddd6d515 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:20.128393 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f01d69-e716-42ba-9244-f8b0ddd6d515 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:20.128671 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f01d69-e716-42ba-9244-f8b0ddd6d515 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:20.128940 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f01d69-e716-42ba-9244-f8b0ddd6d515 None admin] PATH_INFO: `/v3/users` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:20.129655 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f01d69-e716-42ba-9244-f8b0ddd6d515 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:59:20.129954 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f01d69-e716-42ba-9244-f8b0ddd6d515 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:20.131378 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f01d69-e716-42ba-9244-f8b0ddd6d515 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:20.154059 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 302/582] 10.222.0.55 () {56 vars in 1137 bytes} [Fri Sep 23 14:59:19 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:59:20.154059 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f01d69-e716-42ba-9244-f8b0ddd6d515 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:20.167798 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eda53d7-6929-4dcd-ba69-819a0d5cb3aa None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:20.190841 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:20.202831 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72aa2484-54ac-4713-ab5a-ab9028e49b0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:20.203552 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72aa2484-54ac-4713-ab5a-ab9028e49b0e None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:20.203841 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72aa2484-54ac-4713-ab5a-ab9028e49b0e None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:20.204156 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72aa2484-54ac-4713-ab5a-ab9028e49b0e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:20.204912 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72aa2484-54ac-4713-ab5a-ab9028e49b0e None admin] RBAC: Adding query filter params (role.id=0665869b71fa40b0b552efdd57faf023, scope.project.id=87b3e9d4b6474bd4a04ba59ef52bf3e2, user.id=2228c0b49a0249bbaf524a8c4e9e108b) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:59:20.205225 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72aa2484-54ac-4713-ab5a-ab9028e49b0e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:20.206691 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72aa2484-54ac-4713-ab5a-ab9028e49b0e None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:20.212699 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 281/583] 10.222.0.55 () {56 vars in 1374 bytes} [Fri Sep 23 14:59:20 2022] GET /identity/v3/role_assignments?user.id=2228c0b49a0249bbaf524a8c4e9e108b&scope.project.id=87b3e9d4b6474bd4a04ba59ef52bf3e2&role.id=0665869b71fa40b0b552efdd57faf023 => generated 259 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:59:20.213792 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72aa2484-54ac-4713-ab5a-ab9028e49b0e None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:20.267755 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f01d69-e716-42ba-9244-f8b0ddd6d515 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:20.284589 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:20.309729 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a9049a-e4f4-4659-aa82-8e27cca4f7c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:20.310377 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a9049a-e4f4-4659-aa82-8e27cca4f7c4 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:20.310663 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a9049a-e4f4-4659-aa82-8e27cca4f7c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:20.310936 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a9049a-e4f4-4659-aa82-8e27cca4f7c4 None admin] PATH_INFO: `/v3/domains` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:20.311639 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a9049a-e4f4-4659-aa82-8e27cca4f7c4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:59:20.311948 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a9049a-e4f4-4659-aa82-8e27cca4f7c4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:20.313377 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a9049a-e4f4-4659-aa82-8e27cca4f7c4 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:20.332423 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 303/584] 10.222.0.55 () {56 vars in 1109 bytes} [Fri Sep 23 14:59:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:59:20.332423 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a9049a-e4f4-4659-aa82-8e27cca4f7c4 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:20.340962 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72aa2484-54ac-4713-ab5a-ab9028e49b0e None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:20.381034 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d797a9-17bf-4e81-a1ff-2f064702cbe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:20.381657 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d797a9-17bf-4e81-a1ff-2f064702cbe9 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:20.381941 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d797a9-17bf-4e81-a1ff-2f064702cbe9 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:20.382210 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d797a9-17bf-4e81-a1ff-2f064702cbe9 None admin] PATH_INFO: `/v3/projects` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:20.382906 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d797a9-17bf-4e81-a1ff-2f064702cbe9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:59:20.383210 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d797a9-17bf-4e81-a1ff-2f064702cbe9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:20.385415 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d797a9-17bf-4e81-a1ff-2f064702cbe9 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:20.404659 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 282/585] 10.222.0.55 () {56 vars in 1148 bytes} [Fri Sep 23 14:59:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:59:20.405508 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d797a9-17bf-4e81-a1ff-2f064702cbe9 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:20.452395 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a9049a-e4f4-4659-aa82-8e27cca4f7c4 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:20.454018 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:20.488803 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b29b82-64d0-4708-9874-c847435146dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:20.489502 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b29b82-64d0-4708-9874-c847435146dc None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:20.489782 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b29b82-64d0-4708-9874-c847435146dc None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:20.490105 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b29b82-64d0-4708-9874-c847435146dc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:20.501400 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b29b82-64d0-4708-9874-c847435146dc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:20.502843 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b29b82-64d0-4708-9874-c847435146dc None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:20.508397 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d797a9-17bf-4e81-a1ff-2f064702cbe9 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:20.518991 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-f7b29b82-64d0-4708-9874-c847435146dc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 14:59:20.518991 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 14:59:20.518991 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 14:59:20.518991 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 14:59:20.518991 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 14:59:20.518991 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 14:59:20.518991 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 14:59:20.518991 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 14:59:20.518991 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 14:59:20.518991 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 14:59:20.518991 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 14:59:20.518991 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 14:59:20.518991 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 14:59:20.518991 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 14:59:20.518991 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 14:59:20.518991 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 14:59:20.519994 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 14:59:20.519994 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 14:59:20.519994 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 14:59:20.519994 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 14:59:20.519994 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 14:59:20.519994 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 14:59:20.519994 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 14:59:20.519994 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Sep 23 14:59:20.519994 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 14:59:20.519994 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Sep 23 14:59:20.519994 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 14:59:20.519994 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 14:59:20.519994 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 14:59:20.519994 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Sep 23 14:59:20.519994 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 14:59:20.519994 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Sep 23 14:59:20.519994 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 14:59:20.521576 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 14:59:20.521576 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 14:59:20.521576 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Sep 23 14:59:20.521576 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 304/586] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Sep 23 14:59:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 14:59:20.521576 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b29b82-64d0-4708-9874-c847435146dc None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:20.549380 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:20.551901 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a523af-e69b-4c3c-b212-93d00615c308 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:20.552663 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a523af-e69b-4c3c-b212-93d00615c308 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:20.553036 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a523af-e69b-4c3c-b212-93d00615c308 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:20.553425 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a523af-e69b-4c3c-b212-93d00615c308 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:20.564555 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a523af-e69b-4c3c-b212-93d00615c308 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:20.566028 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a523af-e69b-4c3c-b212-93d00615c308 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:20.581208 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application [None req-e1a523af-e69b-4c3c-b212-93d00615c308 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 14:59:20.581208 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 14:59:20.581208 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 14:59:20.581208 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 14:59:20.581208 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 14:59:20.581208 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 14:59:20.581208 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 14:59:20.581208 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 14:59:20.581208 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 14:59:20.581208 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 14:59:20.581208 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 14:59:20.581208 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 14:59:20.581208 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 14:59:20.581208 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 14:59:20.581208 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 14:59:20.581208 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 14:59:20.582193 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 14:59:20.582193 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 14:59:20.582193 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 14:59:20.582193 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 14:59:20.582193 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 14:59:20.582193 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 14:59:20.582193 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 14:59:20.582193 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application with Lock( Sep 23 14:59:20.582193 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 14:59:20.582193 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._enter() Sep 23 14:59:20.582193 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 14:59:20.582193 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 14:59:20.582193 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 14:59:20.582193 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.creator() Sep 23 14:59:20.582193 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 14:59:20.582193 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application created_value = creator( Sep 23 14:59:20.582193 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 14:59:20.583766 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 14:59:20.583766 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 14:59:20.583766 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Sep 23 14:59:20.583766 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 283/587] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Sep 23 14:59:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 14:59:20.583766 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a523af-e69b-4c3c-b212-93d00615c308 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:20.639261 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b29b82-64d0-4708-9874-c847435146dc None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:20.678704 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22f2492-5f33-42e3-960d-70a2e56c4c87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:20.679441 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22f2492-5f33-42e3-960d-70a2e56c4c87 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:20.679732 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22f2492-5f33-42e3-960d-70a2e56c4c87 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:20.680099 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22f2492-5f33-42e3-960d-70a2e56c4c87 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:20.689087 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22f2492-5f33-42e3-960d-70a2e56c4c87 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:20.690553 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22f2492-5f33-42e3-960d-70a2e56c4c87 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:20.695322 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:20.700037 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a523af-e69b-4c3c-b212-93d00615c308 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:20.708696 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-d22f2492-5f33-42e3-960d-70a2e56c4c87 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 23 14:59:20.708696 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 14:59:20.708696 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 14:59:20.708696 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 14:59:20.708696 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 14:59:20.708696 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 14:59:20.708696 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 14:59:20.708696 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 14:59:20.708696 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 14:59:20.708696 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 14:59:20.708696 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 14:59:20.708696 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 14:59:20.708696 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 23 14:59:20.708696 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 23 14:59:20.708696 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 23 14:59:20.708696 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 23 14:59:20.710473 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 14:59:20.710473 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 14:59:20.710473 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 23 14:59:20.710473 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 14:59:20.710473 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 23 14:59:20.710473 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 14:59:20.710473 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 14:59:20.710473 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 14:59:20.710473 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 14:59:20.710473 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 14:59:20.710473 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 14:59:20.710473 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Sep 23 14:59:20.710473 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 14:59:20.710473 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Sep 23 14:59:20.710473 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 14:59:20.710473 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 14:59:20.710473 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 14:59:20.710473 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Sep 23 14:59:20.712875 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 14:59:20.712875 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Sep 23 14:59:20.712875 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 23 14:59:20.712875 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 23 14:59:20.712875 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 23 14:59:20.712875 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 23 14:59:20.712875 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 23 14:59:20.712875 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 23 14:59:20.712875 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 23 14:59:20.712875 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Sep 23 14:59:20.712875 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 305/588] 10.222.0.55 () {56 vars in 1099 bytes} [Fri Sep 23 14:59:20 2022] GET /identity/v3/users/cinder => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 14:59:20.712875 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22f2492-5f33-42e3-960d-70a2e56c4c87 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:20.740739 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8b6ad6-2c4a-4c3e-8c2d-3c090c0e83c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:20.741746 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8b6ad6-2c4a-4c3e-8c2d-3c090c0e83c4 None admin] REQUEST_METHOD: `PUT` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:20.742331 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8b6ad6-2c4a-4c3e-8c2d-3c090c0e83c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:20.742811 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8b6ad6-2c4a-4c3e-8c2d-3c090c0e83c4 None admin] PATH_INFO: `/v3/projects/87b3e9d4b6474bd4a04ba59ef52bf3e2/users/b3c6c7a1325a47be924268e1aa1a7a18/roles/0665869b71fa40b0b552efdd57faf023` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:20.760881 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8b6ad6-2c4a-4c3e-8c2d-3c090c0e83c4 None admin] RBAC: Authorizing `identity:create_grant(project_id=87b3e9d4b6474bd4a04ba59ef52bf3e2, user_id=b3c6c7a1325a47be924268e1aa1a7a18, role_id=0665869b71fa40b0b552efdd57faf023)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:20.762641 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8b6ad6-2c4a-4c3e-8c2d-3c090c0e83c4 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:20.804257 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:20.820057 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22f2492-5f33-42e3-960d-70a2e56c4c87 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:20.854314 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab89aa92-405d-4682-bf09-b3166980f305 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:20.854921 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab89aa92-405d-4682-bf09-b3166980f305 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:20.855204 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab89aa92-405d-4682-bf09-b3166980f305 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:20.855477 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab89aa92-405d-4682-bf09-b3166980f305 None admin] PATH_INFO: `/v3/domains` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:20.856186 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab89aa92-405d-4682-bf09-b3166980f305 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:59:20.856495 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab89aa92-405d-4682-bf09-b3166980f305 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:20.857853 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab89aa92-405d-4682-bf09-b3166980f305 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:20.869914 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 306/589] 10.222.0.55 () {56 vars in 1109 bytes} [Fri Sep 23 14:59:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:59:20.870718 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab89aa92-405d-4682-bf09-b3166980f305 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:20.972032 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab89aa92-405d-4682-bf09-b3166980f305 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:20.984078 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 284/590] 10.222.0.55 () {58 vars in 1439 bytes} [Fri Sep 23 14:59:20 2022] PUT /identity/v3/projects/87b3e9d4b6474bd4a04ba59ef52bf3e2/users/b3c6c7a1325a47be924268e1aa1a7a18/roles/0665869b71fa40b0b552efdd57faf023 => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 14:59:20.985023 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8b6ad6-2c4a-4c3e-8c2d-3c090c0e83c4 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:21.018245 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c55369-2244-4123-9c6c-47b8dceca916 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:21.019369 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c55369-2244-4123-9c6c-47b8dceca916 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:21.019913 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c55369-2244-4123-9c6c-47b8dceca916 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:21.020835 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c55369-2244-4123-9c6c-47b8dceca916 None admin] PATH_INFO: `/v3/domains` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:21.022003 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c55369-2244-4123-9c6c-47b8dceca916 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:59:21.022513 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c55369-2244-4123-9c6c-47b8dceca916 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:21.025586 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c55369-2244-4123-9c6c-47b8dceca916 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:21.047376 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 307/591] 10.222.0.55 () {56 vars in 1109 bytes} [Fri Sep 23 14:59:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:59:21.047376 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c55369-2244-4123-9c6c-47b8dceca916 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:21.065612 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:21.137249 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8b6ad6-2c4a-4c3e-8c2d-3c090c0e83c4 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:21.175642 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c55369-2244-4123-9c6c-47b8dceca916 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:21.178209 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb3f034-4bfd-469b-bc97-ba1f7caf78fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:21.179034 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb3f034-4bfd-469b-bc97-ba1f7caf78fa None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:21.179310 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb3f034-4bfd-469b-bc97-ba1f7caf78fa None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:21.179579 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb3f034-4bfd-469b-bc97-ba1f7caf78fa None admin] PATH_INFO: `/v3/users` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:21.182479 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb3f034-4bfd-469b-bc97-ba1f7caf78fa None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:59:21.182479 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb3f034-4bfd-469b-bc97-ba1f7caf78fa None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:21.182479 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb3f034-4bfd-469b-bc97-ba1f7caf78fa None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:21.204619 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 285/592] 10.222.0.55 () {56 vars in 1137 bytes} [Fri Sep 23 14:59:20 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:59:21.205855 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb3f034-4bfd-469b-bc97-ba1f7caf78fa None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:21.217739 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:21.218871 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b65fdc3-6629-4bca-8890-55f96eafd7c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:21.219512 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b65fdc3-6629-4bca-8890-55f96eafd7c2 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:21.219797 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b65fdc3-6629-4bca-8890-55f96eafd7c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:21.220476 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b65fdc3-6629-4bca-8890-55f96eafd7c2 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:21.228748 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b65fdc3-6629-4bca-8890-55f96eafd7c2 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:21.254938 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b65fdc3-6629-4bca-8890-55f96eafd7c2 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:21.254938 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-6b65fdc3-6629-4bca-8890-55f96eafd7c2 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 23 14:59:21.254938 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 14:59:21.254938 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 14:59:21.254938 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 14:59:21.254938 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 14:59:21.254938 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 14:59:21.254938 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 14:59:21.254938 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 14:59:21.254938 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 14:59:21.254938 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 14:59:21.254938 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 14:59:21.254938 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 14:59:21.254938 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 23 14:59:21.255784 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 23 14:59:21.255784 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 23 14:59:21.255784 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 23 14:59:21.255784 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 14:59:21.255784 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 14:59:21.255784 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 23 14:59:21.255784 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 14:59:21.255784 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 23 14:59:21.255784 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 14:59:21.255784 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 14:59:21.255784 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 14:59:21.255784 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 14:59:21.255784 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 14:59:21.255784 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 14:59:21.255784 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Sep 23 14:59:21.255784 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 14:59:21.255784 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Sep 23 14:59:21.255784 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 14:59:21.256796 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 14:59:21.256796 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 14:59:21.256796 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Sep 23 14:59:21.256796 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 14:59:21.256796 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Sep 23 14:59:21.256796 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 23 14:59:21.256796 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 23 14:59:21.256796 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 23 14:59:21.256796 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 23 14:59:21.256796 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 23 14:59:21.256796 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 23 14:59:21.256796 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 23 14:59:21.256796 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Sep 23 14:59:21.256796 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 308/593] 10.222.0.55 () {56 vars in 1102 bytes} [Fri Sep 23 14:59:21 2022] GET /identity/v3/users/neutron => generated 85 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 14:59:21.256796 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b65fdc3-6629-4bca-8890-55f96eafd7c2 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:21.321156 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:21.340132 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb3f034-4bfd-469b-bc97-ba1f7caf78fa None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:21.369603 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b65fdc3-6629-4bca-8890-55f96eafd7c2 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:21.386651 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c938cc88-d716-45e9-a77d-bf5f26b05c33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:21.387390 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c938cc88-d716-45e9-a77d-bf5f26b05c33 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:21.387740 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c938cc88-d716-45e9-a77d-bf5f26b05c33 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:21.388123 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c938cc88-d716-45e9-a77d-bf5f26b05c33 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:21.397006 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c938cc88-d716-45e9-a77d-bf5f26b05c33 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:21.399364 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c938cc88-d716-45e9-a77d-bf5f26b05c33 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:21.411085 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5f4176-b5ae-4bc6-bfe2-251b9e9f7107 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:21.412053 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5f4176-b5ae-4bc6-bfe2-251b9e9f7107 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:21.412516 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5f4176-b5ae-4bc6-bfe2-251b9e9f7107 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:21.412966 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5f4176-b5ae-4bc6-bfe2-251b9e9f7107 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:21.415173 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application [None req-c938cc88-d716-45e9-a77d-bf5f26b05c33 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 14:59:21.415173 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 14:59:21.415173 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 14:59:21.415173 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 14:59:21.415173 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 14:59:21.415173 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 14:59:21.415173 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 14:59:21.415173 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 14:59:21.415173 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 14:59:21.415173 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 14:59:21.415173 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 14:59:21.415173 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 14:59:21.415173 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 23 14:59:21.415173 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 23 14:59:21.415173 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 23 14:59:21.416110 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 23 14:59:21.416110 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 14:59:21.416110 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 14:59:21.416110 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 14:59:21.416110 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 14:59:21.416110 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 14:59:21.416110 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 14:59:21.416110 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 14:59:21.416110 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application with Lock( Sep 23 14:59:21.416110 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 14:59:21.416110 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._enter() Sep 23 14:59:21.416110 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 14:59:21.416110 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 14:59:21.416110 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 14:59:21.416110 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.creator() Sep 23 14:59:21.416110 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 14:59:21.416110 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application created_value = creator( Sep 23 14:59:21.417166 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 23 14:59:21.417166 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 23 14:59:21.417166 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 23 14:59:21.417166 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 23 14:59:21.417166 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 23 14:59:21.417166 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 23 14:59:21.417166 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 14:59:21.417166 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Sep 23 14:59:21.417768 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 286/594] 10.222.0.55 () {56 vars in 1111 bytes} [Fri Sep 23 14:59:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 14:59:21.418429 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c938cc88-d716-45e9-a77d-bf5f26b05c33 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:21.429505 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5f4176-b5ae-4bc6-bfe2-251b9e9f7107 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:21.432219 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5f4176-b5ae-4bc6-bfe2-251b9e9f7107 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:21.450032 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-8d5f4176-b5ae-4bc6-bfe2-251b9e9f7107 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 14:59:21.450032 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 14:59:21.450032 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 14:59:21.450032 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 14:59:21.450032 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 14:59:21.450032 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 14:59:21.450032 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 14:59:21.450032 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 14:59:21.450032 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 14:59:21.450032 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 14:59:21.450032 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 14:59:21.450032 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 14:59:21.450032 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 14:59:21.450032 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 14:59:21.450032 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 14:59:21.450032 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 14:59:21.451217 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 14:59:21.451217 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 14:59:21.451217 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 14:59:21.451217 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 14:59:21.451217 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 14:59:21.451217 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 14:59:21.451217 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 14:59:21.451217 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Sep 23 14:59:21.451217 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 14:59:21.451217 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Sep 23 14:59:21.451217 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 14:59:21.451217 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 14:59:21.451217 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 14:59:21.451217 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Sep 23 14:59:21.451217 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 14:59:21.451217 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Sep 23 14:59:21.451217 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 14:59:21.452346 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 14:59:21.452346 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 14:59:21.452346 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Sep 23 14:59:21.454884 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 309/595] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Sep 23 14:59:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 14:59:21.456103 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5f4176-b5ae-4bc6-bfe2-251b9e9f7107 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:21.520445 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:21.552587 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c938cc88-d716-45e9-a77d-bf5f26b05c33 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:21.584832 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5f4176-b5ae-4bc6-bfe2-251b9e9f7107 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:21.592102 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff19cf2-35c8-4a41-b3fa-3c96c135e033 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:21.592950 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff19cf2-35c8-4a41-b3fa-3c96c135e033 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:21.593763 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff19cf2-35c8-4a41-b3fa-3c96c135e033 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:21.594187 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff19cf2-35c8-4a41-b3fa-3c96c135e033 None admin] PATH_INFO: `/v3/users` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:21.595122 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff19cf2-35c8-4a41-b3fa-3c96c135e033 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:59:21.596080 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff19cf2-35c8-4a41-b3fa-3c96c135e033 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:21.598271 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff19cf2-35c8-4a41-b3fa-3c96c135e033 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:21.604258 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:21.621408 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 287/596] 10.222.0.55 () {56 vars in 1139 bytes} [Fri Sep 23 14:59:21 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:59:21.622981 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff19cf2-35c8-4a41-b3fa-3c96c135e033 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:21.626057 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd06d3d3-e981-4a58-9ce7-34e7af770d4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:21.627231 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd06d3d3-e981-4a58-9ce7-34e7af770d4a None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:21.627618 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd06d3d3-e981-4a58-9ce7-34e7af770d4a None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:21.628010 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd06d3d3-e981-4a58-9ce7-34e7af770d4a None admin] PATH_INFO: `/v3/projects` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:21.629676 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd06d3d3-e981-4a58-9ce7-34e7af770d4a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:59:21.630149 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd06d3d3-e981-4a58-9ce7-34e7af770d4a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:21.631756 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd06d3d3-e981-4a58-9ce7-34e7af770d4a None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:21.648505 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 310/597] 10.222.0.55 () {56 vars in 1148 bytes} [Fri Sep 23 14:59:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:59:21.650064 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd06d3d3-e981-4a58-9ce7-34e7af770d4a None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:21.761385 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff19cf2-35c8-4a41-b3fa-3c96c135e033 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:21.778710 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd06d3d3-e981-4a58-9ce7-34e7af770d4a None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:21.797267 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:21.813642 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56ae48b-edca-46b9-a7a7-3da474d8ca45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:21.814506 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56ae48b-edca-46b9-a7a7-3da474d8ca45 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:21.815171 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56ae48b-edca-46b9-a7a7-3da474d8ca45 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:21.816004 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56ae48b-edca-46b9-a7a7-3da474d8ca45 None admin] PATH_INFO: `/v3/domains` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:21.817084 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56ae48b-edca-46b9-a7a7-3da474d8ca45 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:59:21.817547 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56ae48b-edca-46b9-a7a7-3da474d8ca45 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:21.819136 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56ae48b-edca-46b9-a7a7-3da474d8ca45 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:21.821091 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccf2fb2-c1e2-4efd-8814-b48c6ded0cec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:21.822719 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccf2fb2-c1e2-4efd-8814-b48c6ded0cec None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:21.822719 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccf2fb2-c1e2-4efd-8814-b48c6ded0cec None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:21.823064 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccf2fb2-c1e2-4efd-8814-b48c6ded0cec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:21.834410 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccf2fb2-c1e2-4efd-8814-b48c6ded0cec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:21.840126 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccf2fb2-c1e2-4efd-8814-b48c6ded0cec None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:21.840256 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 288/598] 10.222.0.55 () {56 vars in 1109 bytes} [Fri Sep 23 14:59:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:59:21.841045 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56ae48b-edca-46b9-a7a7-3da474d8ca45 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:21.855935 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-2ccf2fb2-c1e2-4efd-8814-b48c6ded0cec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 14:59:21.855935 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 14:59:21.855935 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 14:59:21.855935 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 14:59:21.855935 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 14:59:21.855935 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 14:59:21.855935 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 14:59:21.855935 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 14:59:21.855935 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 14:59:21.855935 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 14:59:21.855935 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 14:59:21.855935 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 14:59:21.855935 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 14:59:21.855935 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 14:59:21.855935 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 14:59:21.855935 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 14:59:21.856949 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 14:59:21.856949 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 14:59:21.856949 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 14:59:21.856949 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 14:59:21.856949 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 14:59:21.856949 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 14:59:21.856949 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 14:59:21.856949 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Sep 23 14:59:21.856949 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 14:59:21.856949 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Sep 23 14:59:21.856949 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 14:59:21.856949 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 14:59:21.856949 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 14:59:21.856949 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Sep 23 14:59:21.856949 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 14:59:21.856949 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Sep 23 14:59:21.856949 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 14:59:21.857837 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 14:59:21.857837 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 14:59:21.857837 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Sep 23 14:59:21.862077 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 311/599] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Sep 23 14:59:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 14:59:21.863034 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccf2fb2-c1e2-4efd-8814-b48c6ded0cec None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:21.894621 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:21.972530 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56ae48b-edca-46b9-a7a7-3da474d8ca45 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:21.984577 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccf2fb2-c1e2-4efd-8814-b48c6ded0cec None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:22.020533 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1847b880-2483-4346-aadf-fcc1b691c133 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:22.021642 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1847b880-2483-4346-aadf-fcc1b691c133 None admin] REQUEST_METHOD: `PUT` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:22.022349 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1847b880-2483-4346-aadf-fcc1b691c133 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:22.022790 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1847b880-2483-4346-aadf-fcc1b691c133 None admin] PATH_INFO: `/v3/projects/87b3e9d4b6474bd4a04ba59ef52bf3e2/users/51c39f1d05634589a88c0b6477bd014e/roles/0665869b71fa40b0b552efdd57faf023` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:22.033100 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2751851f-c062-4653-9081-616f6b653fc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:22.033892 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2751851f-c062-4653-9081-616f6b653fc2 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:22.034276 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2751851f-c062-4653-9081-616f6b653fc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:22.034670 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2751851f-c062-4653-9081-616f6b653fc2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:22.042374 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2751851f-c062-4653-9081-616f6b653fc2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:22.043944 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2751851f-c062-4653-9081-616f6b653fc2 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:22.045179 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1847b880-2483-4346-aadf-fcc1b691c133 None admin] RBAC: Authorizing `identity:create_grant(project_id=87b3e9d4b6474bd4a04ba59ef52bf3e2, user_id=51c39f1d05634589a88c0b6477bd014e, role_id=0665869b71fa40b0b552efdd57faf023)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:22.047450 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1847b880-2483-4346-aadf-fcc1b691c133 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:22.056943 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-2751851f-c062-4653-9081-616f6b653fc2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 14:59:22.056943 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 14:59:22.056943 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 14:59:22.056943 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 14:59:22.056943 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 14:59:22.056943 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 14:59:22.056943 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 14:59:22.056943 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 14:59:22.056943 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 14:59:22.056943 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 14:59:22.056943 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 14:59:22.056943 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 14:59:22.056943 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 23 14:59:22.056943 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 23 14:59:22.056943 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 23 14:59:22.057878 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 23 14:59:22.057878 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 14:59:22.057878 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 14:59:22.057878 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 14:59:22.057878 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 14:59:22.057878 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 14:59:22.057878 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 14:59:22.057878 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 14:59:22.057878 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Sep 23 14:59:22.057878 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 14:59:22.057878 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Sep 23 14:59:22.057878 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 14:59:22.057878 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 14:59:22.057878 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 14:59:22.057878 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Sep 23 14:59:22.057878 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 14:59:22.057878 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Sep 23 14:59:22.058751 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 23 14:59:22.058751 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 23 14:59:22.058751 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 23 14:59:22.058751 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 23 14:59:22.058751 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 23 14:59:22.058751 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 23 14:59:22.058751 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 14:59:22.058751 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Sep 23 14:59:22.064934 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 312/600] 10.222.0.55 () {56 vars in 1111 bytes} [Fri Sep 23 14:59:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 14:59:22.064934 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2751851f-c062-4653-9081-616f6b653fc2 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:22.154119 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 289/601] 10.222.0.55 () {58 vars in 1439 bytes} [Fri Sep 23 14:59:21 2022] PUT /identity/v3/projects/87b3e9d4b6474bd4a04ba59ef52bf3e2/users/51c39f1d05634589a88c0b6477bd014e/roles/0665869b71fa40b0b552efdd57faf023 => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 14:59:22.154119 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1847b880-2483-4346-aadf-fcc1b691c133 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:22.180396 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:22.203913 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2751851f-c062-4653-9081-616f6b653fc2 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:22.228854 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:22.276050 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a42354-e843-4c5f-805e-652e3737bc0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:22.276687 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a42354-e843-4c5f-805e-652e3737bc0f None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:22.276970 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a42354-e843-4c5f-805e-652e3737bc0f None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:22.277244 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a42354-e843-4c5f-805e-652e3737bc0f None admin] PATH_INFO: `/v3/domains` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:22.277901 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a42354-e843-4c5f-805e-652e3737bc0f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:59:22.278206 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a42354-e843-4c5f-805e-652e3737bc0f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:22.279885 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a42354-e843-4c5f-805e-652e3737bc0f None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:22.309332 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 313/602] 10.222.0.55 () {56 vars in 1109 bytes} [Fri Sep 23 14:59:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:59:22.319679 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a42354-e843-4c5f-805e-652e3737bc0f None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:22.383387 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1847b880-2483-4346-aadf-fcc1b691c133 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:22.436300 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128153ba-9112-4db6-934f-35cd59e5435e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:22.436837 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128153ba-9112-4db6-934f-35cd59e5435e None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:22.437137 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128153ba-9112-4db6-934f-35cd59e5435e None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:22.437396 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128153ba-9112-4db6-934f-35cd59e5435e None admin] PATH_INFO: `/v3/projects` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:22.438112 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128153ba-9112-4db6-934f-35cd59e5435e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:59:22.438414 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128153ba-9112-4db6-934f-35cd59e5435e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:22.440205 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128153ba-9112-4db6-934f-35cd59e5435e None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:22.456629 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 290/603] 10.222.0.55 () {56 vars in 1148 bytes} [Fri Sep 23 14:59:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:59:22.458682 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc22a98-81f1-43bf-8173-c40443459a95 None None] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:22.458975 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc22a98-81f1-43bf-8173-c40443459a95 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:22.459270 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc22a98-81f1-43bf-8173-c40443459a95 None None] PATH_INFO: `/` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:22.465460 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 291/604] 10.222.0.55 () {54 vars in 927 bytes} [Fri Sep 23 14:59:22 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 14:59:22.467276 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc22a98-81f1-43bf-8173-c40443459a95 None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:22.478710 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a42354-e843-4c5f-805e-652e3737bc0f None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:22.516791 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7200a3-4aea-42a6-922b-3eec56f2c63b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:22.517538 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7200a3-4aea-42a6-922b-3eec56f2c63b None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:22.517815 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7200a3-4aea-42a6-922b-3eec56f2c63b None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:22.518088 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7200a3-4aea-42a6-922b-3eec56f2c63b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:22.526337 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7200a3-4aea-42a6-922b-3eec56f2c63b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:22.527777 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7200a3-4aea-42a6-922b-3eec56f2c63b None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:22.540836 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-0c7200a3-4aea-42a6-922b-3eec56f2c63b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 14:59:22.540836 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 14:59:22.540836 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 14:59:22.540836 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 14:59:22.540836 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 14:59:22.540836 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 14:59:22.540836 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 14:59:22.540836 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 14:59:22.540836 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 14:59:22.540836 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 14:59:22.540836 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 14:59:22.540836 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 14:59:22.540836 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 23 14:59:22.540836 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 23 14:59:22.540836 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 23 14:59:22.541930 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 23 14:59:22.541930 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 14:59:22.541930 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 14:59:22.541930 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 14:59:22.541930 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 14:59:22.541930 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 14:59:22.541930 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 14:59:22.541930 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 14:59:22.541930 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Sep 23 14:59:22.541930 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 14:59:22.541930 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Sep 23 14:59:22.541930 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 14:59:22.541930 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 14:59:22.541930 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 14:59:22.541930 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Sep 23 14:59:22.541930 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 14:59:22.541930 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Sep 23 14:59:22.543012 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 23 14:59:22.543012 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 23 14:59:22.543012 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 23 14:59:22.543012 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 23 14:59:22.543012 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 23 14:59:22.543012 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 23 14:59:22.543012 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 14:59:22.543012 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Sep 23 14:59:22.543012 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 314/605] 10.222.0.55 () {56 vars in 1111 bytes} [Fri Sep 23 14:59:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 14:59:22.543563 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c2cc70-8fde-4f86-8ac8-88dcdd5d48da None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:22.543876 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c2cc70-8fde-4f86-8ac8-88dcdd5d48da None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:22.548615 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c2cc70-8fde-4f86-8ac8-88dcdd5d48da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:22.567473 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-85c2cc70-8fde-4f86-8ac8-88dcdd5d48da None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 14:59:22.568809 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-85c2cc70-8fde-4f86-8ac8-88dcdd5d48da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 14:59:22.571140 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:22.584433 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:22.594769 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc22a98-81f1-43bf-8173-c40443459a95 None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:22.633187 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97ac9ab-0001-43eb-9c2f-2e3a13b9689e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:22.634725 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97ac9ab-0001-43eb-9c2f-2e3a13b9689e None admin] REQUEST_METHOD: `PUT` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:22.635193 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97ac9ab-0001-43eb-9c2f-2e3a13b9689e None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:22.635612 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97ac9ab-0001-43eb-9c2f-2e3a13b9689e None admin] PATH_INFO: `/v3/projects/87b3e9d4b6474bd4a04ba59ef52bf3e2/users/feca6278be904ef8b0dc580a67e477f8/roles/0665869b71fa40b0b552efdd57faf023` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:22.651570 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97ac9ab-0001-43eb-9c2f-2e3a13b9689e None admin] RBAC: Authorizing `identity:create_grant(project_id=87b3e9d4b6474bd4a04ba59ef52bf3e2, user_id=feca6278be904ef8b0dc580a67e477f8, role_id=0665869b71fa40b0b552efdd57faf023)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:22.653283 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97ac9ab-0001-43eb-9c2f-2e3a13b9689e None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:22.654476 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 315/606] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Sep 23 14:59:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 14:59:22.655382 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c2cc70-8fde-4f86-8ac8-88dcdd5d48da None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:22.762433 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c2cc70-8fde-4f86-8ac8-88dcdd5d48da None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:22.778054 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 292/607] 10.222.0.55 () {58 vars in 1439 bytes} [Fri Sep 23 14:59:22 2022] PUT /identity/v3/projects/87b3e9d4b6474bd4a04ba59ef52bf3e2/users/feca6278be904ef8b0dc580a67e477f8/roles/0665869b71fa40b0b552efdd57faf023 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 14:59:22.805644 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6915be-7823-4471-904c-ba19ef7dff4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:22.806878 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6915be-7823-4471-904c-ba19ef7dff4c None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:22.807343 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6915be-7823-4471-904c-ba19ef7dff4c None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:22.807623 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6915be-7823-4471-904c-ba19ef7dff4c None admin] PATH_INFO: `/v3/projects` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:22.808360 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6915be-7823-4471-904c-ba19ef7dff4c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:59:22.808667 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6915be-7823-4471-904c-ba19ef7dff4c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:22.810103 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6915be-7823-4471-904c-ba19ef7dff4c None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:22.822310 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:22.827885 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 316/608] 10.222.0.55 () {56 vars in 1148 bytes} [Fri Sep 23 14:59:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:59:22.833736 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97ac9ab-0001-43eb-9c2f-2e3a13b9689e None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:22.936773 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6915be-7823-4471-904c-ba19ef7dff4c None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:22.990295 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97ac9ab-0001-43eb-9c2f-2e3a13b9689e None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:23.051142 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c832aa2-53c1-4e30-ad4f-2a61b964cec8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:23.051743 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c832aa2-53c1-4e30-ad4f-2a61b964cec8 None admin] REQUEST_METHOD: `PUT` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:23.052044 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c832aa2-53c1-4e30-ad4f-2a61b964cec8 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:23.052322 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c832aa2-53c1-4e30-ad4f-2a61b964cec8 None admin] PATH_INFO: `/v3/projects/87b3e9d4b6474bd4a04ba59ef52bf3e2/users/d259178570864ffeb936ee10dfec6b28/roles/0665869b71fa40b0b552efdd57faf023` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:23.072804 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c832aa2-53c1-4e30-ad4f-2a61b964cec8 None admin] RBAC: Authorizing `identity:create_grant(project_id=87b3e9d4b6474bd4a04ba59ef52bf3e2, user_id=d259178570864ffeb936ee10dfec6b28, role_id=0665869b71fa40b0b552efdd57faf023)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:23.074790 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c832aa2-53c1-4e30-ad4f-2a61b964cec8 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:23.077340 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:23.097792 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6915be-7823-4471-904c-ba19ef7dff4c None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:23.134335 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d746703-2f76-4738-8845-947740123144 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:23.135419 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d746703-2f76-4738-8845-947740123144 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:23.135698 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d746703-2f76-4738-8845-947740123144 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:23.135967 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d746703-2f76-4738-8845-947740123144 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:23.147349 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d746703-2f76-4738-8845-947740123144 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 14:59:23.153492 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-2d746703-2f76-4738-8845-947740123144 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 23 14:59:23.153492 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 14:59:23.153492 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 14:59:23.153492 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 14:59:23.153492 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 14:59:23.153492 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 14:59:23.153492 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 14:59:23.153492 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 14:59:23.153492 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 14:59:23.153492 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 14:59:23.153492 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 14:59:23.153492 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 14:59:23.153492 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 14:59:23.153492 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 14:59:23.153492 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 14:59:23.153492 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 14:59:23.154630 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 14:59:23.154630 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 14:59:23.154630 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 14:59:23.154630 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 14:59:23.154630 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 14:59:23.154630 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 14:59:23.154630 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 14:59:23.154630 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 14:59:23.154630 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 14:59:23.154630 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 14:59:23.154630 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 14:59:23.154630 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Sep 23 14:59:23.154630 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 14:59:23.154630 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Sep 23 14:59:23.154630 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 14:59:23.154630 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 14:59:23.155667 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 14:59:23.155667 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Sep 23 14:59:23.155667 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 14:59:23.155667 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Sep 23 14:59:23.155667 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 14:59:23.155667 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 14:59:23.155667 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 14:59:23.155667 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 14:59:23.155667 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 14:59:23.155667 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 14:59:23.155667 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 23 14:59:23.155667 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Sep 23 14:59:23.155667 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 317/609] 10.222.0.55 () {56 vars in 1102 bytes} [Fri Sep 23 14:59:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 14:59:23.161939 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d746703-2f76-4738-8845-947740123144 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:23.193060 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 293/610] 10.222.0.55 () {58 vars in 1439 bytes} [Fri Sep 23 14:59:22 2022] PUT /identity/v3/projects/87b3e9d4b6474bd4a04ba59ef52bf3e2/users/d259178570864ffeb936ee10dfec6b28/roles/0665869b71fa40b0b552efdd57faf023 => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 14:59:23.235459 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:23.304718 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d746703-2f76-4738-8845-947740123144 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:23.364725 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ffb922-da5f-4f0a-b86e-a484c666d855 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:23.365369 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ffb922-da5f-4f0a-b86e-a484c666d855 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:23.365648 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ffb922-da5f-4f0a-b86e-a484c666d855 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:23.365919 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ffb922-da5f-4f0a-b86e-a484c666d855 None admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:23.366606 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ffb922-da5f-4f0a-b86e-a484c666d855 None admin] RBAC: Adding query filter params (name=service) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:59:23.366909 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ffb922-da5f-4f0a-b86e-a484c666d855 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:23.369189 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ffb922-da5f-4f0a-b86e-a484c666d855 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:23.385579 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 318/611] 10.222.0.55 () {56 vars in 1103 bytes} [Fri Sep 23 14:59:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:59:23.391293 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c832aa2-53c1-4e30-ad4f-2a61b964cec8 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:23.442672 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b3b526-2dfe-4bce-b437-1edf05d713b2 None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:23.443019 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b3b526-2dfe-4bce-b437-1edf05d713b2 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:23.443249 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b3b526-2dfe-4bce-b437-1edf05d713b2 None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:23.444632 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 319/612] 10.222.0.55 () {54 vars in 927 bytes} [Fri Sep 23 14:59:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 23 14:59:23.452209 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a876d7-c404-4b27-a004-354f71d8e5fe None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:23.452491 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a876d7-c404-4b27-a004-354f71d8e5fe None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:23.452806 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a876d7-c404-4b27-a004-354f71d8e5fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:23.476126 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-42a876d7-c404-4b27-a004-354f71d8e5fe None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 14:59:23.478185 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-42a876d7-c404-4b27-a004-354f71d8e5fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 14:59:23.526723 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c832aa2-53c1-4e30-ad4f-2a61b964cec8 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:23.565716 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 320/613] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Sep 23 14:59:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 23 14:59:23.577482 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2368179c-58a7-43bd-8393-ee6c437e7c48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:23.578133 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2368179c-58a7-43bd-8393-ee6c437e7c48 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:23.578410 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2368179c-58a7-43bd-8393-ee6c437e7c48 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:23.578683 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2368179c-58a7-43bd-8393-ee6c437e7c48 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:23.590832 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2368179c-58a7-43bd-8393-ee6c437e7c48 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:23.592488 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2368179c-58a7-43bd-8393-ee6c437e7c48 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:23.607864 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application [None req-2368179c-58a7-43bd-8393-ee6c437e7c48 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 14:59:23.607864 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 14:59:23.607864 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 14:59:23.607864 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 14:59:23.607864 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 14:59:23.607864 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 14:59:23.607864 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 14:59:23.607864 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 14:59:23.607864 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 14:59:23.607864 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 14:59:23.607864 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 14:59:23.607864 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 14:59:23.607864 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 14:59:23.607864 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 14:59:23.607864 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 14:59:23.607864 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 14:59:23.609887 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 14:59:23.609887 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 14:59:23.609887 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 14:59:23.609887 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 14:59:23.609887 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 14:59:23.609887 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 14:59:23.609887 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 14:59:23.609887 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application with Lock( Sep 23 14:59:23.609887 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 14:59:23.609887 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._enter() Sep 23 14:59:23.609887 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 14:59:23.609887 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 14:59:23.609887 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 14:59:23.609887 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.creator() Sep 23 14:59:23.609887 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 14:59:23.609887 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application created_value = creator( Sep 23 14:59:23.609887 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 14:59:23.610999 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 14:59:23.610999 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 14:59:23.610999 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Sep 23 14:59:23.610999 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 294/614] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Sep 23 14:59:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 14:59:23.614994 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a876d7-c404-4b27-a004-354f71d8e5fe None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:23.686967 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:23.747242 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a876d7-c404-4b27-a004-354f71d8e5fe None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:23.783926 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b8b8de-069c-452b-97f2-27d840f05af7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:23.784663 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b8b8de-069c-452b-97f2-27d840f05af7 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:23.784942 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b8b8de-069c-452b-97f2-27d840f05af7 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:23.785213 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b8b8de-069c-452b-97f2-27d840f05af7 None admin] PATH_INFO: `/v3/domains` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:23.785854 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b8b8de-069c-452b-97f2-27d840f05af7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:59:23.786150 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b8b8de-069c-452b-97f2-27d840f05af7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:23.787510 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b8b8de-069c-452b-97f2-27d840f05af7 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:23.800011 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 321/615] 10.222.0.55 () {56 vars in 1109 bytes} [Fri Sep 23 14:59:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:59:23.805156 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2368179c-58a7-43bd-8393-ee6c437e7c48 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:23.893215 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:23.927312 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b8b8de-069c-452b-97f2-27d840f05af7 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:23.979671 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2368179c-58a7-43bd-8393-ee6c437e7c48 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:24.045927 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13ece5f-20fc-4be5-8e4a-5fccee140d36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:24.046963 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13ece5f-20fc-4be5-8e4a-5fccee140d36 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:24.047448 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13ece5f-20fc-4be5-8e4a-5fccee140d36 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:24.047937 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13ece5f-20fc-4be5-8e4a-5fccee140d36 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:24.061077 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13ece5f-20fc-4be5-8e4a-5fccee140d36 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:24.063586 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13ece5f-20fc-4be5-8e4a-5fccee140d36 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:24.070879 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b8b8de-069c-452b-97f2-27d840f05af7 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:24.092375 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:24.094051 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application [None req-e13ece5f-20fc-4be5-8e4a-5fccee140d36 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 23 14:59:24.094051 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 14:59:24.094051 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 14:59:24.094051 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 14:59:24.094051 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 14:59:24.094051 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 14:59:24.094051 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 14:59:24.094051 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 14:59:24.094051 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 14:59:24.094051 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 14:59:24.094051 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 14:59:24.094051 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 14:59:24.094051 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 23 14:59:24.094051 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 23 14:59:24.094051 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 23 14:59:24.094051 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 23 14:59:24.095000 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 14:59:24.095000 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 14:59:24.095000 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 23 14:59:24.095000 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 14:59:24.095000 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 23 14:59:24.095000 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 14:59:24.095000 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 14:59:24.095000 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 14:59:24.095000 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 14:59:24.095000 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 14:59:24.095000 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 14:59:24.095000 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application with Lock( Sep 23 14:59:24.095000 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 14:59:24.095000 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._enter() Sep 23 14:59:24.095000 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 14:59:24.095000 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 14:59:24.095000 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 14:59:24.095000 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.creator() Sep 23 14:59:24.095953 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 14:59:24.095953 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application created_value = creator( Sep 23 14:59:24.095953 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 23 14:59:24.095953 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 23 14:59:24.095953 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 23 14:59:24.095953 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 23 14:59:24.095953 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 23 14:59:24.095953 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 23 14:59:24.095953 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 23 14:59:24.095953 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Sep 23 14:59:24.100015 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 295/616] 10.222.0.55 () {56 vars in 1096 bytes} [Fri Sep 23 14:59:23 2022] GET /identity/v3/users/swift => generated 83 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 14:59:24.101944 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13ece5f-20fc-4be5-8e4a-5fccee140d36 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:24.111902 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667215b2-1ebf-409d-b86f-b331b1a4cc9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:24.112635 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667215b2-1ebf-409d-b86f-b331b1a4cc9b None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:24.112917 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667215b2-1ebf-409d-b86f-b331b1a4cc9b None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:24.113191 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667215b2-1ebf-409d-b86f-b331b1a4cc9b None admin] PATH_INFO: `/v3/roles/service` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:24.126837 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667215b2-1ebf-409d-b86f-b331b1a4cc9b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 14:59:24.133484 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-667215b2-1ebf-409d-b86f-b331b1a4cc9b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 23 14:59:24.133484 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 14:59:24.133484 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 14:59:24.133484 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 14:59:24.133484 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 14:59:24.133484 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 14:59:24.133484 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 14:59:24.133484 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 14:59:24.133484 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 14:59:24.133484 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 14:59:24.133484 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 14:59:24.133484 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 14:59:24.133484 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 14:59:24.133484 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 14:59:24.133484 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 14:59:24.133484 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 14:59:24.136320 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 14:59:24.136320 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 14:59:24.136320 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 14:59:24.136320 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 14:59:24.136320 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 14:59:24.136320 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 14:59:24.136320 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 14:59:24.136320 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 14:59:24.136320 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 14:59:24.136320 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 14:59:24.136320 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 14:59:24.136320 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Sep 23 14:59:24.136320 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 14:59:24.136320 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Sep 23 14:59:24.136320 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 14:59:24.136320 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 14:59:24.138594 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 14:59:24.138594 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Sep 23 14:59:24.138594 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 14:59:24.138594 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Sep 23 14:59:24.138594 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 14:59:24.138594 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 14:59:24.138594 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 14:59:24.138594 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 14:59:24.138594 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 14:59:24.138594 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 14:59:24.138594 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 23 14:59:24.138594 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Sep 23 14:59:24.147138 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 322/617] 10.222.0.55 () {56 vars in 1102 bytes} [Fri Sep 23 14:59:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 23 14:59:24.147138 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667215b2-1ebf-409d-b86f-b331b1a4cc9b None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:24.280343 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13ece5f-20fc-4be5-8e4a-5fccee140d36 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:24.289371 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:24.297325 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667215b2-1ebf-409d-b86f-b331b1a4cc9b None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:24.343134 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f635d2-9049-4d03-916d-857de324ec32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:24.343134 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f635d2-9049-4d03-916d-857de324ec32 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:24.344396 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f635d2-9049-4d03-916d-857de324ec32 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:24.344396 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f635d2-9049-4d03-916d-857de324ec32 None admin] PATH_INFO: `/v3/users` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:24.344396 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f635d2-9049-4d03-916d-857de324ec32 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:59:24.344866 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f635d2-9049-4d03-916d-857de324ec32 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:24.346337 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f635d2-9049-4d03-916d-857de324ec32 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:24.365186 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 296/618] 10.222.0.55 () {56 vars in 1135 bytes} [Fri Sep 23 14:59:24 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 14:59:24.371147 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f635d2-9049-4d03-916d-857de324ec32 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:24.375502 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afaa0cde-a40c-4e22-aa90-480894ad7ad3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:24.376947 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaa0cde-a40c-4e22-aa90-480894ad7ad3 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:24.377228 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaa0cde-a40c-4e22-aa90-480894ad7ad3 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:24.377501 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaa0cde-a40c-4e22-aa90-480894ad7ad3 None admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:24.378165 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afaa0cde-a40c-4e22-aa90-480894ad7ad3 None admin] RBAC: Adding query filter params (name=service) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:59:24.378466 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afaa0cde-a40c-4e22-aa90-480894ad7ad3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:24.379815 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afaa0cde-a40c-4e22-aa90-480894ad7ad3 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:24.396260 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 323/619] 10.222.0.55 () {56 vars in 1103 bytes} [Fri Sep 23 14:59:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:59:24.402649 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afaa0cde-a40c-4e22-aa90-480894ad7ad3 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:24.475246 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:24.568990 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f635d2-9049-4d03-916d-857de324ec32 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:24.584130 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afaa0cde-a40c-4e22-aa90-480894ad7ad3 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:24.648693 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdc4150-765c-4a77-afdb-919683de9168 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:24.649859 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdc4150-765c-4a77-afdb-919683de9168 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:24.650350 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdc4150-765c-4a77-afdb-919683de9168 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:24.650712 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdc4150-765c-4a77-afdb-919683de9168 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:24.663080 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85848a4-9b26-483e-9eec-4ad4747f8cd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:24.663653 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85848a4-9b26-483e-9eec-4ad4747f8cd7 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:24.663934 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85848a4-9b26-483e-9eec-4ad4747f8cd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:24.664230 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85848a4-9b26-483e-9eec-4ad4747f8cd7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:24.676121 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdc4150-765c-4a77-afdb-919683de9168 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:24.677605 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdc4150-765c-4a77-afdb-919683de9168 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:24.683899 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85848a4-9b26-483e-9eec-4ad4747f8cd7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:24.685366 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85848a4-9b26-483e-9eec-4ad4747f8cd7 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:24.706815 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application [None req-0bdc4150-765c-4a77-afdb-919683de9168 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 14:59:24.706815 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 14:59:24.706815 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 14:59:24.706815 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 14:59:24.706815 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 14:59:24.706815 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 14:59:24.706815 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 14:59:24.706815 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 14:59:24.706815 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 14:59:24.706815 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 14:59:24.706815 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 14:59:24.706815 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 14:59:24.706815 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 14:59:24.706815 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 14:59:24.706815 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 14:59:24.706815 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 14:59:24.707954 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 14:59:24.707954 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 14:59:24.707954 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 14:59:24.707954 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 14:59:24.707954 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 14:59:24.707954 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 14:59:24.707954 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 14:59:24.707954 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application with Lock( Sep 23 14:59:24.707954 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 14:59:24.707954 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._enter() Sep 23 14:59:24.707954 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 14:59:24.707954 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 14:59:24.707954 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 14:59:24.707954 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.creator() Sep 23 14:59:24.707954 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 14:59:24.707954 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application created_value = creator( Sep 23 14:59:24.707954 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 14:59:24.708939 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 14:59:24.708939 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 14:59:24.708939 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Sep 23 14:59:24.717170 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 297/620] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Sep 23 14:59:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 346 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 14:59:24.718140 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdc4150-765c-4a77-afdb-919683de9168 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:24.721246 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-a85848a4-9b26-483e-9eec-4ad4747f8cd7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 14:59:24.721246 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 14:59:24.721246 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 14:59:24.721246 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 14:59:24.721246 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 14:59:24.721246 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 14:59:24.721246 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 14:59:24.721246 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 14:59:24.721246 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 14:59:24.721246 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 14:59:24.721246 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 14:59:24.721246 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 14:59:24.721246 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 14:59:24.721246 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 14:59:24.721246 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 14:59:24.721246 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 14:59:24.723511 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 14:59:24.723511 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 14:59:24.723511 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 14:59:24.723511 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 14:59:24.723511 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 14:59:24.723511 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 14:59:24.723511 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 14:59:24.723511 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Sep 23 14:59:24.723511 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 14:59:24.723511 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Sep 23 14:59:24.723511 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 14:59:24.723511 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 14:59:24.723511 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 14:59:24.723511 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Sep 23 14:59:24.723511 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 14:59:24.723511 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Sep 23 14:59:24.723511 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 14:59:24.725756 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 14:59:24.725756 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 14:59:24.725756 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Sep 23 14:59:24.725756 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 324/621] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Sep 23 14:59:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 320 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 14:59:24.733491 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85848a4-9b26-483e-9eec-4ad4747f8cd7 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:24.752870 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:24.843245 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdc4150-765c-4a77-afdb-919683de9168 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:24.858092 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:24.879495 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85848a4-9b26-483e-9eec-4ad4747f8cd7 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:24.879827 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc43eec-b7ba-435e-be0b-a041232d8f77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:24.880487 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc43eec-b7ba-435e-be0b-a041232d8f77 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:24.880768 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc43eec-b7ba-435e-be0b-a041232d8f77 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:24.881044 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc43eec-b7ba-435e-be0b-a041232d8f77 None admin] PATH_INFO: `/v3/domains` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:24.881687 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc43eec-b7ba-435e-be0b-a041232d8f77 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:59:24.881989 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc43eec-b7ba-435e-be0b-a041232d8f77 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:24.883871 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc43eec-b7ba-435e-be0b-a041232d8f77 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:24.896233 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 298/622] 10.222.0.55 () {56 vars in 1109 bytes} [Fri Sep 23 14:59:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:59:24.901978 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62604620-3844-4c2a-a013-0fcae93497f6 None None] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:24.902271 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62604620-3844-4c2a-a013-0fcae93497f6 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:24.902545 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62604620-3844-4c2a-a013-0fcae93497f6 None None] PATH_INFO: `/` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:24.908306 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 299/623] 10.222.0.55 () {54 vars in 927 bytes} [Fri Sep 23 14:59:24 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 14:59:24.908306 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62604620-3844-4c2a-a013-0fcae93497f6 None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:24.933032 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfb1628-4537-4606-8974-3913f2aff030 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:24.935027 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfb1628-4537-4606-8974-3913f2aff030 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:24.935517 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfb1628-4537-4606-8974-3913f2aff030 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:24.936364 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfb1628-4537-4606-8974-3913f2aff030 None admin] PATH_INFO: `/v3/domains` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:24.937661 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfb1628-4537-4606-8974-3913f2aff030 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:59:24.938401 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfb1628-4537-4606-8974-3913f2aff030 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:24.940681 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfb1628-4537-4606-8974-3913f2aff030 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:24.961399 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 325/624] 10.222.0.55 () {56 vars in 1109 bytes} [Fri Sep 23 14:59:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:59:24.961587 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7f8a58-d48c-49b7-879b-91b16138e47e None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:24.962505 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7f8a58-d48c-49b7-879b-91b16138e47e None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:24.968005 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7f8a58-d48c-49b7-879b-91b16138e47e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:24.987255 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-2f7f8a58-d48c-49b7-879b-91b16138e47e None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 14:59:24.988523 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-2f7f8a58-d48c-49b7-879b-91b16138e47e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 14:59:25.002775 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:25.034545 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62604620-3844-4c2a-a013-0fcae93497f6 None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:25.084749 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957948ae-ae21-4788-9921-79510fc81045 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:25.086140 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957948ae-ae21-4788-9921-79510fc81045 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:25.087025 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957948ae-ae21-4788-9921-79510fc81045 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:25.088142 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957948ae-ae21-4788-9921-79510fc81045 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:25.088801 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 326/625] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Sep 23 14:59:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 14:59:25.089476 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7f8a58-d48c-49b7-879b-91b16138e47e None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:25.108448 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957948ae-ae21-4788-9921-79510fc81045 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:25.112142 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957948ae-ae21-4788-9921-79510fc81045 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:25.140021 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application [None req-957948ae-ae21-4788-9921-79510fc81045 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 14:59:25.140021 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 14:59:25.140021 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 14:59:25.140021 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 14:59:25.140021 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 14:59:25.140021 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 14:59:25.140021 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 14:59:25.140021 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 14:59:25.140021 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 14:59:25.140021 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 14:59:25.140021 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 14:59:25.140021 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 14:59:25.140021 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 23 14:59:25.140021 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 23 14:59:25.140021 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 23 14:59:25.141005 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 23 14:59:25.141005 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 14:59:25.141005 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 14:59:25.141005 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 14:59:25.141005 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 14:59:25.141005 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 14:59:25.141005 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 14:59:25.141005 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 14:59:25.141005 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application with Lock( Sep 23 14:59:25.141005 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 14:59:25.141005 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._enter() Sep 23 14:59:25.141005 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 14:59:25.141005 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 14:59:25.141005 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 14:59:25.141005 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.creator() Sep 23 14:59:25.141005 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 14:59:25.141005 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application created_value = creator( Sep 23 14:59:25.141970 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 23 14:59:25.141970 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 23 14:59:25.141970 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 23 14:59:25.141970 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 23 14:59:25.141970 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 23 14:59:25.141970 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 23 14:59:25.141970 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 14:59:25.141970 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Sep 23 14:59:25.149383 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 300/626] 10.222.0.55 () {56 vars in 1111 bytes} [Fri Sep 23 14:59:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 14:59:25.163670 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957948ae-ae21-4788-9921-79510fc81045 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:25.205775 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:25.233441 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7f8a58-d48c-49b7-879b-91b16138e47e None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:25.274003 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc6adfd-b823-4d54-82a4-a236fe47ebe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:25.275441 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc6adfd-b823-4d54-82a4-a236fe47ebe2 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:25.275947 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc6adfd-b823-4d54-82a4-a236fe47ebe2 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:25.276486 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc6adfd-b823-4d54-82a4-a236fe47ebe2 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:25.284771 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc6adfd-b823-4d54-82a4-a236fe47ebe2 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:25.286517 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc6adfd-b823-4d54-82a4-a236fe47ebe2 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:25.308799 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-8cc6adfd-b823-4d54-82a4-a236fe47ebe2 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 23 14:59:25.308799 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 14:59:25.308799 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 14:59:25.308799 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 14:59:25.308799 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 14:59:25.308799 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 14:59:25.308799 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 14:59:25.308799 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 14:59:25.308799 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 14:59:25.308799 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 14:59:25.308799 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 14:59:25.308799 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 14:59:25.308799 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 23 14:59:25.308799 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 23 14:59:25.308799 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 23 14:59:25.308799 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 23 14:59:25.309828 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 14:59:25.309828 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 14:59:25.309828 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 23 14:59:25.309828 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 14:59:25.309828 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 23 14:59:25.309828 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 14:59:25.309828 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 14:59:25.309828 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 14:59:25.309828 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 14:59:25.309828 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 14:59:25.309828 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 14:59:25.309828 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Sep 23 14:59:25.309828 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 14:59:25.309828 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Sep 23 14:59:25.309828 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 14:59:25.309828 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 14:59:25.309828 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 14:59:25.309828 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Sep 23 14:59:25.310784 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 14:59:25.310784 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Sep 23 14:59:25.310784 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 23 14:59:25.310784 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 23 14:59:25.310784 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 23 14:59:25.310784 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 23 14:59:25.310784 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 23 14:59:25.310784 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 23 14:59:25.310784 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 23 14:59:25.310784 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Sep 23 14:59:25.317842 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 327/627] 10.222.0.55 () {56 vars in 1093 bytes} [Fri Sep 23 14:59:25 2022] GET /identity/v3/users/nova => generated 82 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 14:59:25.327779 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc6adfd-b823-4d54-82a4-a236fe47ebe2 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:25.382812 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957948ae-ae21-4788-9921-79510fc81045 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:25.420642 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de7d46f-7fc4-4f11-b88c-baf80a8b98b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:25.421368 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de7d46f-7fc4-4f11-b88c-baf80a8b98b1 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:25.421644 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de7d46f-7fc4-4f11-b88c-baf80a8b98b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:25.421955 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de7d46f-7fc4-4f11-b88c-baf80a8b98b1 None admin] PATH_INFO: `/v3/projects` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:25.422698 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de7d46f-7fc4-4f11-b88c-baf80a8b98b1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:59:25.422938 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de7d46f-7fc4-4f11-b88c-baf80a8b98b1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:25.424382 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de7d46f-7fc4-4f11-b88c-baf80a8b98b1 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:25.426943 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:25.438199 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 301/628] 10.222.0.55 () {56 vars in 1148 bytes} [Fri Sep 23 14:59:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 14:59:25.439589 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de7d46f-7fc4-4f11-b88c-baf80a8b98b1 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:25.471499 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc6adfd-b823-4d54-82a4-a236fe47ebe2 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:25.511607 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0864a12d-08f1-41cd-877f-c29bbec8f180 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:25.512612 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0864a12d-08f1-41cd-877f-c29bbec8f180 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:25.513155 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0864a12d-08f1-41cd-877f-c29bbec8f180 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:25.513631 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0864a12d-08f1-41cd-877f-c29bbec8f180 None admin] PATH_INFO: `/v3/users` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:25.514563 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0864a12d-08f1-41cd-877f-c29bbec8f180 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:59:25.515081 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0864a12d-08f1-41cd-877f-c29bbec8f180 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:25.516749 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0864a12d-08f1-41cd-877f-c29bbec8f180 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:25.518781 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:25.537498 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 328/629] 10.222.0.55 () {56 vars in 1133 bytes} [Fri Sep 23 14:59:25 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:59:25.538653 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0864a12d-08f1-41cd-877f-c29bbec8f180 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:25.549024 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de7d46f-7fc4-4f11-b88c-baf80a8b98b1 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:25.592538 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225c3117-f1c6-44f8-ad78-ff22e18f7118 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:25.593176 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225c3117-f1c6-44f8-ad78-ff22e18f7118 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:25.593478 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225c3117-f1c6-44f8-ad78-ff22e18f7118 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:25.593711 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225c3117-f1c6-44f8-ad78-ff22e18f7118 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:25.606054 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225c3117-f1c6-44f8-ad78-ff22e18f7118 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 14:59:25.612899 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application [None req-225c3117-f1c6-44f8-ad78-ff22e18f7118 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 23 14:59:25.612899 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 14:59:25.612899 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 14:59:25.612899 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 14:59:25.612899 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 14:59:25.612899 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 14:59:25.612899 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 14:59:25.612899 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 14:59:25.612899 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 14:59:25.612899 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 14:59:25.612899 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 14:59:25.612899 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 14:59:25.612899 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 14:59:25.612899 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 14:59:25.612899 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 14:59:25.612899 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 14:59:25.613833 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 14:59:25.613833 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 14:59:25.613833 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 14:59:25.613833 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 14:59:25.613833 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 14:59:25.613833 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 14:59:25.613833 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 14:59:25.613833 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 14:59:25.613833 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 14:59:25.613833 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 14:59:25.613833 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 14:59:25.613833 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application with Lock( Sep 23 14:59:25.613833 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 14:59:25.613833 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._enter() Sep 23 14:59:25.613833 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 14:59:25.613833 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 14:59:25.614721 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 14:59:25.614721 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.creator() Sep 23 14:59:25.614721 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 14:59:25.614721 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application created_value = creator( Sep 23 14:59:25.614721 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 14:59:25.614721 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 14:59:25.614721 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 14:59:25.614721 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 14:59:25.614721 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 14:59:25.614721 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 14:59:25.614721 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 23 14:59:25.614721 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Sep 23 14:59:25.614721 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 302/630] 10.222.0.55 () {56 vars in 1102 bytes} [Fri Sep 23 14:59:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 14:59:25.615432 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225c3117-f1c6-44f8-ad78-ff22e18f7118 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:25.653967 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0864a12d-08f1-41cd-877f-c29bbec8f180 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:25.681424 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:25.693406 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004e535c-f840-4f64-93ae-d8fb691fd606 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:25.694131 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004e535c-f840-4f64-93ae-d8fb691fd606 None admin] REQUEST_METHOD: `PUT` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:25.694415 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004e535c-f840-4f64-93ae-d8fb691fd606 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:25.694734 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004e535c-f840-4f64-93ae-d8fb691fd606 None admin] PATH_INFO: `/v3/projects/87b3e9d4b6474bd4a04ba59ef52bf3e2/users/2228c0b49a0249bbaf524a8c4e9e108b/roles/0665869b71fa40b0b552efdd57faf023` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:25.711524 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004e535c-f840-4f64-93ae-d8fb691fd606 None admin] RBAC: Authorizing `identity:create_grant(project_id=87b3e9d4b6474bd4a04ba59ef52bf3e2, user_id=2228c0b49a0249bbaf524a8c4e9e108b, role_id=0665869b71fa40b0b552efdd57faf023)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:25.713052 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004e535c-f840-4f64-93ae-d8fb691fd606 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:25.729138 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225c3117-f1c6-44f8-ad78-ff22e18f7118 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:25.762353 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3bd8f7-e991-438c-8e7e-fd7ce0fb465e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:25.763049 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3bd8f7-e991-438c-8e7e-fd7ce0fb465e None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:25.763395 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3bd8f7-e991-438c-8e7e-fd7ce0fb465e None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:25.763719 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3bd8f7-e991-438c-8e7e-fd7ce0fb465e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:25.800197 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 329/631] 10.222.0.55 () {58 vars in 1439 bytes} [Fri Sep 23 14:59:25 2022] PUT /identity/v3/projects/87b3e9d4b6474bd4a04ba59ef52bf3e2/users/2228c0b49a0249bbaf524a8c4e9e108b/roles/0665869b71fa40b0b552efdd57faf023 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 14:59:25.800197 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3f2216-4a92-4f49-a595-23bd662474d0 None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:25.800197 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3f2216-4a92-4f49-a595-23bd662474d0 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:25.800197 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3f2216-4a92-4f49-a595-23bd662474d0 None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:25.800197 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 330/632] 10.222.0.55 () {54 vars in 927 bytes} [Fri Sep 23 14:59:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 14:59:25.800197 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3f2216-4a92-4f49-a595-23bd662474d0 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:25.849387 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3bd8f7-e991-438c-8e7e-fd7ce0fb465e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:25.851047 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3bd8f7-e991-438c-8e7e-fd7ce0fb465e None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:25.861365 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:25.935226 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3f2216-4a92-4f49-a595-23bd662474d0 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:25.940433 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:25.989795 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5abfa2-0bea-42fe-950c-00cdcfbd7bd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:25.990896 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5abfa2-0bea-42fe-950c-00cdcfbd7bd6 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:25.991759 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5abfa2-0bea-42fe-950c-00cdcfbd7bd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:25.992245 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5abfa2-0bea-42fe-950c-00cdcfbd7bd6 None admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:25.993650 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5abfa2-0bea-42fe-950c-00cdcfbd7bd6 None admin] RBAC: Adding query filter params (name=service) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:59:25.994426 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5abfa2-0bea-42fe-950c-00cdcfbd7bd6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:25.996991 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5abfa2-0bea-42fe-950c-00cdcfbd7bd6 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:25.999406 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 303/633] 10.222.0.55 () {58 vars in 1282 bytes} [Fri Sep 23 14:59:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 23 14:59:26.001536 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b690ebd-b318-4011-b134-7ae00f752753 None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:26.001918 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b690ebd-b318-4011-b134-7ae00f752753 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:26.002280 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b690ebd-b318-4011-b134-7ae00f752753 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:26.007666 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 331/634] 10.222.0.55 () {56 vars in 1103 bytes} [Fri Sep 23 14:59:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:59:26.012254 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5abfa2-0bea-42fe-950c-00cdcfbd7bd6 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:26.024275 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-1b690ebd-b318-4011-b134-7ae00f752753 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 14:59:26.026220 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-1b690ebd-b318-4011-b134-7ae00f752753 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 14:59:26.127288 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 304/635] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Sep 23 14:59:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 14:59:26.128552 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b690ebd-b318-4011-b134-7ae00f752753 None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:26.153337 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5abfa2-0bea-42fe-950c-00cdcfbd7bd6 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:26.156670 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:26.194858 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359e9611-a44a-410f-891f-ad4b183b84c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:26.195502 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359e9611-a44a-410f-891f-ad4b183b84c2 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:26.195786 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359e9611-a44a-410f-891f-ad4b183b84c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:26.196105 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359e9611-a44a-410f-891f-ad4b183b84c2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:26.209618 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359e9611-a44a-410f-891f-ad4b183b84c2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:26.211391 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359e9611-a44a-410f-891f-ad4b183b84c2 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:26.228842 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-359e9611-a44a-410f-891f-ad4b183b84c2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 14:59:26.228842 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 14:59:26.228842 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 14:59:26.228842 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 14:59:26.228842 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 14:59:26.228842 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 14:59:26.228842 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 14:59:26.228842 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 14:59:26.228842 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 14:59:26.228842 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 14:59:26.228842 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 14:59:26.228842 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 14:59:26.228842 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 14:59:26.228842 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 14:59:26.228842 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 14:59:26.228842 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 14:59:26.229957 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 14:59:26.229957 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 14:59:26.229957 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 14:59:26.229957 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 14:59:26.229957 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 14:59:26.229957 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 14:59:26.229957 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 14:59:26.229957 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Sep 23 14:59:26.229957 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 14:59:26.229957 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Sep 23 14:59:26.229957 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 14:59:26.229957 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 14:59:26.229957 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 14:59:26.229957 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Sep 23 14:59:26.229957 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 14:59:26.229957 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Sep 23 14:59:26.229957 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 14:59:26.230850 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 14:59:26.230850 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 14:59:26.230850 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Sep 23 14:59:26.230850 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 332/636] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Sep 23 14:59:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 14:59:26.231665 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2179cff-2b73-4adf-9bef-6d27c3264821 None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:26.232021 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2179cff-2b73-4adf-9bef-6d27c3264821 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:26.232306 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2179cff-2b73-4adf-9bef-6d27c3264821 None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:26.233833 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 333/637] 10.222.0.55 () {54 vars in 927 bytes} [Fri Sep 23 14:59:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 14:59:26.241015 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c831eb2-7bef-423b-a89f-cce99e837f30 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:26.241258 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c831eb2-7bef-423b-a89f-cce99e837f30 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:26.241495 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c831eb2-7bef-423b-a89f-cce99e837f30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:26.252302 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:26.262577 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-8c831eb2-7bef-423b-a89f-cce99e837f30 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 14:59:26.263596 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-8c831eb2-7bef-423b-a89f-cce99e837f30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 14:59:26.268943 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b690ebd-b318-4011-b134-7ae00f752753 None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:26.313531 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b83698-3434-4fca-9c73-e5e663803fd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:26.314900 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b83698-3434-4fca-9c73-e5e663803fd9 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:26.315338 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b83698-3434-4fca-9c73-e5e663803fd9 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:26.316186 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b83698-3434-4fca-9c73-e5e663803fd9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:26.330547 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b83698-3434-4fca-9c73-e5e663803fd9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:26.333035 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b83698-3434-4fca-9c73-e5e663803fd9 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:26.348013 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 334/638] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Sep 23 14:59:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 23 14:59:26.348013 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c831eb2-7bef-423b-a89f-cce99e837f30 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:26.362947 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application [None req-55b83698-3434-4fca-9c73-e5e663803fd9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 14:59:26.362947 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 14:59:26.362947 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 14:59:26.362947 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 14:59:26.362947 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 14:59:26.362947 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 14:59:26.362947 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 14:59:26.362947 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 14:59:26.362947 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 14:59:26.362947 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 14:59:26.362947 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 14:59:26.362947 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 14:59:26.362947 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 14:59:26.362947 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 14:59:26.362947 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 14:59:26.362947 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 14:59:26.363946 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 14:59:26.363946 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 14:59:26.363946 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 14:59:26.363946 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 14:59:26.363946 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 14:59:26.363946 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 14:59:26.363946 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 14:59:26.363946 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application with Lock( Sep 23 14:59:26.363946 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 14:59:26.363946 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._enter() Sep 23 14:59:26.363946 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 14:59:26.363946 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 14:59:26.363946 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 14:59:26.363946 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.creator() Sep 23 14:59:26.363946 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 14:59:26.363946 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application created_value = creator( Sep 23 14:59:26.363946 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 14:59:26.365283 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 14:59:26.365283 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 14:59:26.365283 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Sep 23 14:59:26.365283 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 305/639] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Sep 23 14:59:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 14:59:26.371159 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b83698-3434-4fca-9c73-e5e663803fd9 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:26.493223 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:26.507170 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b83698-3434-4fca-9c73-e5e663803fd9 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:26.511689 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c831eb2-7bef-423b-a89f-cce99e837f30 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:26.553429 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f038a5-e8f1-4962-a85b-14d0e2251dac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:26.554047 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f038a5-e8f1-4962-a85b-14d0e2251dac None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:26.554323 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f038a5-e8f1-4962-a85b-14d0e2251dac None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:26.554598 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f038a5-e8f1-4962-a85b-14d0e2251dac None admin] PATH_INFO: `/v3/domains` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:26.555241 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f038a5-e8f1-4962-a85b-14d0e2251dac None admin] RBAC: Adding query filter params (name=Default) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:59:26.555546 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f038a5-e8f1-4962-a85b-14d0e2251dac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:26.557987 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f038a5-e8f1-4962-a85b-14d0e2251dac None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:26.562439 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086a60b7-6e71-4591-a416-d60c538b89d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:26.563066 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086a60b7-6e71-4591-a416-d60c538b89d0 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:26.563349 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086a60b7-6e71-4591-a416-d60c538b89d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:26.563625 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086a60b7-6e71-4591-a416-d60c538b89d0 None admin] PATH_INFO: `/v3/domains` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:26.564346 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086a60b7-6e71-4591-a416-d60c538b89d0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:59:26.564669 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086a60b7-6e71-4591-a416-d60c538b89d0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:26.566053 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086a60b7-6e71-4591-a416-d60c538b89d0 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:26.578372 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:26.584548 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 306/641] 10.222.0.55 () {56 vars in 1109 bytes} [Fri Sep 23 14:59:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:59:26.584697 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 335/641] 10.222.0.55 () {56 vars in 1109 bytes} [Fri Sep 23 14:59:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:59:26.585793 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086a60b7-6e71-4591-a416-d60c538b89d0 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:26.587108 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f038a5-e8f1-4962-a85b-14d0e2251dac None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:26.689782 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086a60b7-6e71-4591-a416-d60c538b89d0 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:26.710484 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f038a5-e8f1-4962-a85b-14d0e2251dac None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:26.727413 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33daf3ac-736d-4e23-82f0-dfc853a2c4bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:26.728684 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33daf3ac-736d-4e23-82f0-dfc853a2c4bb None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:26.728972 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33daf3ac-736d-4e23-82f0-dfc853a2c4bb None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:26.729356 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33daf3ac-736d-4e23-82f0-dfc853a2c4bb None admin] PATH_INFO: `/v3/roles/service` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:26.744811 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33daf3ac-736d-4e23-82f0-dfc853a2c4bb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 14:59:26.749611 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application [None req-33daf3ac-736d-4e23-82f0-dfc853a2c4bb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 23 14:59:26.749611 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 14:59:26.749611 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 14:59:26.749611 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 14:59:26.749611 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 14:59:26.749611 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 14:59:26.749611 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 14:59:26.749611 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 14:59:26.749611 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 14:59:26.749611 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 14:59:26.749611 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 14:59:26.749611 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 14:59:26.749611 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 14:59:26.749611 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 14:59:26.749611 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 14:59:26.749611 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 14:59:26.750524 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 14:59:26.750524 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 14:59:26.750524 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 14:59:26.750524 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 14:59:26.750524 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 14:59:26.750524 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 14:59:26.750524 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 14:59:26.750524 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 14:59:26.750524 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 14:59:26.750524 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 14:59:26.750524 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 14:59:26.750524 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application with Lock( Sep 23 14:59:26.750524 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 14:59:26.750524 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._enter() Sep 23 14:59:26.750524 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 14:59:26.750524 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 14:59:26.751378 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 14:59:26.751378 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.creator() Sep 23 14:59:26.751378 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 14:59:26.751378 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application created_value = creator( Sep 23 14:59:26.751378 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 14:59:26.751378 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 14:59:26.751378 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 14:59:26.751378 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 14:59:26.751378 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 14:59:26.751378 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 14:59:26.751378 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 23 14:59:26.751378 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Sep 23 14:59:26.751378 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 307/642] 10.222.0.55 () {56 vars in 1102 bytes} [Fri Sep 23 14:59:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 14:59:26.752771 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33daf3ac-736d-4e23-82f0-dfc853a2c4bb None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:26.757185 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4294d0bb-c02c-4b6f-95f2-35fb39471c90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:26.757826 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4294d0bb-c02c-4b6f-95f2-35fb39471c90 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:26.758107 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4294d0bb-c02c-4b6f-95f2-35fb39471c90 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:26.758377 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4294d0bb-c02c-4b6f-95f2-35fb39471c90 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:26.765669 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4294d0bb-c02c-4b6f-95f2-35fb39471c90 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:26.767174 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4294d0bb-c02c-4b6f-95f2-35fb39471c90 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:26.781174 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-4294d0bb-c02c-4b6f-95f2-35fb39471c90 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 14:59:26.781174 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 14:59:26.781174 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 14:59:26.781174 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 14:59:26.781174 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 14:59:26.781174 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 14:59:26.781174 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 14:59:26.781174 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 14:59:26.781174 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 14:59:26.781174 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 14:59:26.781174 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 14:59:26.781174 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 14:59:26.781174 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 23 14:59:26.781174 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 23 14:59:26.781174 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 23 14:59:26.782575 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 23 14:59:26.782575 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 14:59:26.782575 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 14:59:26.782575 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 14:59:26.782575 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 14:59:26.782575 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 14:59:26.782575 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 14:59:26.782575 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 14:59:26.782575 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Sep 23 14:59:26.782575 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 14:59:26.782575 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Sep 23 14:59:26.782575 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 14:59:26.782575 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 14:59:26.782575 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 14:59:26.782575 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Sep 23 14:59:26.782575 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 14:59:26.782575 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Sep 23 14:59:26.783528 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 23 14:59:26.783528 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 23 14:59:26.783528 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 23 14:59:26.783528 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 23 14:59:26.783528 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 23 14:59:26.783528 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 23 14:59:26.783528 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 14:59:26.783528 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Sep 23 14:59:26.783528 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 336/643] 10.222.0.55 () {56 vars in 1111 bytes} [Fri Sep 23 14:59:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 14:59:26.783528 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4294d0bb-c02c-4b6f-95f2-35fb39471c90 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:26.798709 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:26.821817 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:26.867176 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33daf3ac-736d-4e23-82f0-dfc853a2c4bb None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:26.902926 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a882ec-84b9-41f2-832f-19772525e1d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:26.903564 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a882ec-84b9-41f2-832f-19772525e1d7 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:26.903843 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a882ec-84b9-41f2-832f-19772525e1d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:26.904151 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a882ec-84b9-41f2-832f-19772525e1d7 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:26.906363 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4294d0bb-c02c-4b6f-95f2-35fb39471c90 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:26.911664 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a882ec-84b9-41f2-832f-19772525e1d7 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:26.913136 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a882ec-84b9-41f2-832f-19772525e1d7 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:26.925899 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application [None req-c7a882ec-84b9-41f2-832f-19772525e1d7 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 23 14:59:26.925899 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 14:59:26.925899 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 14:59:26.925899 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 14:59:26.925899 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 14:59:26.925899 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 14:59:26.925899 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 14:59:26.925899 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 14:59:26.925899 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 14:59:26.925899 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 14:59:26.925899 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 14:59:26.925899 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 14:59:26.925899 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 23 14:59:26.925899 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 23 14:59:26.925899 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 23 14:59:26.925899 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 23 14:59:26.926863 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 14:59:26.926863 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 14:59:26.926863 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 23 14:59:26.926863 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 14:59:26.926863 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 23 14:59:26.926863 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 14:59:26.926863 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 14:59:26.926863 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 14:59:26.926863 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 14:59:26.926863 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 14:59:26.926863 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 14:59:26.926863 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application with Lock( Sep 23 14:59:26.926863 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 14:59:26.926863 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._enter() Sep 23 14:59:26.926863 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 14:59:26.926863 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 14:59:26.926863 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 14:59:26.926863 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.creator() Sep 23 14:59:26.927929 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 14:59:26.927929 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application created_value = creator( Sep 23 14:59:26.927929 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 23 14:59:26.927929 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 23 14:59:26.927929 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 23 14:59:26.927929 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 23 14:59:26.927929 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 23 14:59:26.927929 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 23 14:59:26.927929 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 23 14:59:26.927929 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Sep 23 14:59:26.929345 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 308/644] 10.222.0.55 () {56 vars in 1099 bytes} [Fri Sep 23 14:59:26 2022] GET /identity/v3/users/glance => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 14:59:26.929345 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a882ec-84b9-41f2-832f-19772525e1d7 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:26.945574 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61575fe7-de36-4ae7-b8f2-61503cab8445 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:26.946250 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61575fe7-de36-4ae7-b8f2-61503cab8445 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:26.946870 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61575fe7-de36-4ae7-b8f2-61503cab8445 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:26.947167 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61575fe7-de36-4ae7-b8f2-61503cab8445 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:26.959395 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61575fe7-de36-4ae7-b8f2-61503cab8445 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 14:59:26.965160 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-61575fe7-de36-4ae7-b8f2-61503cab8445 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 23 14:59:26.965160 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 14:59:26.965160 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 14:59:26.965160 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 14:59:26.965160 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 14:59:26.965160 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 14:59:26.965160 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 14:59:26.965160 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 14:59:26.965160 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 14:59:26.965160 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 14:59:26.965160 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 14:59:26.965160 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 14:59:26.965160 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 14:59:26.965160 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 14:59:26.965160 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 14:59:26.965160 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 14:59:26.966073 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 14:59:26.966073 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 14:59:26.966073 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 14:59:26.966073 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 14:59:26.966073 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 14:59:26.966073 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 14:59:26.966073 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 14:59:26.966073 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 14:59:26.966073 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 14:59:26.966073 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 14:59:26.966073 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 14:59:26.966073 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Sep 23 14:59:26.966073 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 14:59:26.966073 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Sep 23 14:59:26.966073 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 14:59:26.966073 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 14:59:26.966904 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 14:59:26.966904 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Sep 23 14:59:26.966904 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 14:59:26.966904 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Sep 23 14:59:26.966904 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 14:59:26.966904 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 14:59:26.966904 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 14:59:26.966904 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 14:59:26.966904 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 14:59:26.966904 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 14:59:26.966904 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 23 14:59:26.966904 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Sep 23 14:59:26.966904 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 337/645] 10.222.0.55 () {56 vars in 1102 bytes} [Fri Sep 23 14:59:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 14:59:26.968211 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61575fe7-de36-4ae7-b8f2-61503cab8445 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:27.039651 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:27.049116 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a882ec-84b9-41f2-832f-19772525e1d7 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:27.079025 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:27.082607 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61575fe7-de36-4ae7-b8f2-61503cab8445 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:27.099288 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e002f11a-b878-485a-ac99-ef7c6b0080b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:27.099907 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e002f11a-b878-485a-ac99-ef7c6b0080b9 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:27.100285 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e002f11a-b878-485a-ac99-ef7c6b0080b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:27.100706 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e002f11a-b878-485a-ac99-ef7c6b0080b9 None admin] PATH_INFO: `/v3/roles` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:27.101380 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e002f11a-b878-485a-ac99-ef7c6b0080b9 None admin] RBAC: Adding query filter params (name=service) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:59:27.101715 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e002f11a-b878-485a-ac99-ef7c6b0080b9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:27.103306 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e002f11a-b878-485a-ac99-ef7c6b0080b9 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:27.112399 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 309/646] 10.222.0.55 () {56 vars in 1103 bytes} [Fri Sep 23 14:59:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:59:27.113522 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e002f11a-b878-485a-ac99-ef7c6b0080b9 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:27.128660 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d202ff-b479-49bd-885e-d7f0482c449c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:27.129261 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d202ff-b479-49bd-885e-d7f0482c449c None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:27.129543 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d202ff-b479-49bd-885e-d7f0482c449c None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:27.129818 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d202ff-b479-49bd-885e-d7f0482c449c None admin] PATH_INFO: `/v3/projects` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:27.130517 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d202ff-b479-49bd-885e-d7f0482c449c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:59:27.131158 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d202ff-b479-49bd-885e-d7f0482c449c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:27.132288 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d202ff-b479-49bd-885e-d7f0482c449c None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:27.147335 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 338/647] 10.222.0.55 () {56 vars in 1148 bytes} [Fri Sep 23 14:59:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:59:27.148228 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d202ff-b479-49bd-885e-d7f0482c449c None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:27.235820 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e002f11a-b878-485a-ac99-ef7c6b0080b9 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:27.257333 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d202ff-b479-49bd-885e-d7f0482c449c None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:27.271810 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab7d060-67ca-478d-ae13-487e9bd9628d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:27.272511 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab7d060-67ca-478d-ae13-487e9bd9628d None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:27.272791 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab7d060-67ca-478d-ae13-487e9bd9628d None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:27.273061 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab7d060-67ca-478d-ae13-487e9bd9628d None admin] PATH_INFO: `/v3/users` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:27.273777 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab7d060-67ca-478d-ae13-487e9bd9628d None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:59:27.274078 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab7d060-67ca-478d-ae13-487e9bd9628d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:27.276256 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab7d060-67ca-478d-ae13-487e9bd9628d None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:27.288268 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:27.299711 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 310/648] 10.222.0.55 () {56 vars in 1137 bytes} [Fri Sep 23 14:59:27 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:59:27.299711 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab7d060-67ca-478d-ae13-487e9bd9628d None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:27.301650 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fe2438-c817-406d-a40c-eded53ea5cad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:27.302260 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fe2438-c817-406d-a40c-eded53ea5cad None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:27.302540 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fe2438-c817-406d-a40c-eded53ea5cad None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:27.302808 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fe2438-c817-406d-a40c-eded53ea5cad None admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:27.303473 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fe2438-c817-406d-a40c-eded53ea5cad None admin] RBAC: Adding query filter params (name=service) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:59:27.303776 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fe2438-c817-406d-a40c-eded53ea5cad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:27.305428 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fe2438-c817-406d-a40c-eded53ea5cad None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:27.314330 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 339/649] 10.222.0.55 () {56 vars in 1103 bytes} [Fri Sep 23 14:59:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:59:27.315151 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fe2438-c817-406d-a40c-eded53ea5cad None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:27.344082 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:27.408173 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab7d060-67ca-478d-ae13-487e9bd9628d None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:27.421191 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fe2438-c817-406d-a40c-eded53ea5cad None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:27.444301 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b40014c-b91c-46d4-b78a-f042c356baa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:27.444940 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b40014c-b91c-46d4-b78a-f042c356baa2 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:27.445223 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b40014c-b91c-46d4-b78a-f042c356baa2 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:27.445531 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b40014c-b91c-46d4-b78a-f042c356baa2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:27.457065 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b40014c-b91c-46d4-b78a-f042c356baa2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:27.457246 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e482d640-13f8-423f-8ab7-104fd59487b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:27.457722 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e482d640-13f8-423f-8ab7-104fd59487b8 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:27.458007 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e482d640-13f8-423f-8ab7-104fd59487b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:27.458273 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e482d640-13f8-423f-8ab7-104fd59487b8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:27.458604 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b40014c-b91c-46d4-b78a-f042c356baa2 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:27.459025 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e482d640-13f8-423f-8ab7-104fd59487b8 None admin] RBAC: Adding query filter params (role.id=0665869b71fa40b0b552efdd57faf023, scope.project.id=87b3e9d4b6474bd4a04ba59ef52bf3e2, user.id=b3c6c7a1325a47be924268e1aa1a7a18) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:59:27.459329 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e482d640-13f8-423f-8ab7-104fd59487b8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:27.460799 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e482d640-13f8-423f-8ab7-104fd59487b8 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:27.472092 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 340/650] 10.222.0.55 () {56 vars in 1374 bytes} [Fri Sep 23 14:59:27 2022] GET /identity/v3/role_assignments?user.id=b3c6c7a1325a47be924268e1aa1a7a18&scope.project.id=87b3e9d4b6474bd4a04ba59ef52bf3e2&role.id=0665869b71fa40b0b552efdd57faf023 => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:59:27.472092 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e482d640-13f8-423f-8ab7-104fd59487b8 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:27.474519 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application [None req-0b40014c-b91c-46d4-b78a-f042c356baa2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 14:59:27.474519 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 14:59:27.474519 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 14:59:27.474519 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 14:59:27.474519 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 14:59:27.474519 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 14:59:27.474519 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 14:59:27.474519 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 14:59:27.474519 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 14:59:27.474519 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 14:59:27.474519 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 14:59:27.474519 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 14:59:27.474519 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 14:59:27.474519 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 14:59:27.474519 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 14:59:27.474519 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 14:59:27.475505 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 14:59:27.475505 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 14:59:27.475505 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 14:59:27.475505 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 14:59:27.475505 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 14:59:27.475505 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 14:59:27.475505 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 14:59:27.475505 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application with Lock( Sep 23 14:59:27.475505 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 14:59:27.475505 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._enter() Sep 23 14:59:27.475505 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 14:59:27.475505 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 14:59:27.475505 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 14:59:27.475505 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.creator() Sep 23 14:59:27.475505 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 14:59:27.475505 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application created_value = creator( Sep 23 14:59:27.475505 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 14:59:27.476488 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 14:59:27.476488 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 14:59:27.476488 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Sep 23 14:59:27.480826 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 311/651] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Sep 23 14:59:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 14:59:27.481724 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b40014c-b91c-46d4-b78a-f042c356baa2 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:27.544153 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:27.602230 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:27.614542 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e482d640-13f8-423f-8ab7-104fd59487b8 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:27.617483 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b40014c-b91c-46d4-b78a-f042c356baa2 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:27.660672 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19341af-ccba-49ef-8314-303906f0bc49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:27.661421 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19341af-ccba-49ef-8314-303906f0bc49 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:27.661709 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19341af-ccba-49ef-8314-303906f0bc49 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:27.661877 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a52772-c0ce-4696-8cf4-9026cfbea57c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:27.662466 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19341af-ccba-49ef-8314-303906f0bc49 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:27.662568 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a52772-c0ce-4696-8cf4-9026cfbea57c None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:27.662971 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a52772-c0ce-4696-8cf4-9026cfbea57c None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:27.663077 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a52772-c0ce-4696-8cf4-9026cfbea57c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:27.676633 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a52772-c0ce-4696-8cf4-9026cfbea57c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:27.678100 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a52772-c0ce-4696-8cf4-9026cfbea57c None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:27.695846 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application [None req-e1a52772-c0ce-4696-8cf4-9026cfbea57c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 14:59:27.695846 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 14:59:27.695846 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 14:59:27.695846 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 14:59:27.695846 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 14:59:27.695846 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 14:59:27.695846 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 14:59:27.695846 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 14:59:27.695846 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 14:59:27.695846 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 14:59:27.695846 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 14:59:27.695846 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 14:59:27.695846 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 14:59:27.695846 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 14:59:27.695846 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 14:59:27.695846 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 14:59:27.703944 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 14:59:27.703944 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 14:59:27.703944 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 14:59:27.703944 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 14:59:27.703944 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 14:59:27.703944 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 14:59:27.703944 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 14:59:27.703944 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application with Lock( Sep 23 14:59:27.703944 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 14:59:27.703944 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._enter() Sep 23 14:59:27.703944 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 14:59:27.703944 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 14:59:27.703944 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 14:59:27.703944 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.creator() Sep 23 14:59:27.703944 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 14:59:27.703944 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application created_value = creator( Sep 23 14:59:27.703944 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 14:59:27.704913 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 14:59:27.704913 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 14:59:27.704913 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Sep 23 14:59:27.704913 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 312/652] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Sep 23 14:59:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 14:59:27.704913 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a52772-c0ce-4696-8cf4-9026cfbea57c None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:27.756347 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19341af-ccba-49ef-8314-303906f0bc49 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:27.758195 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19341af-ccba-49ef-8314-303906f0bc49 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:27.816698 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a52772-c0ce-4696-8cf4-9026cfbea57c None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:27.824435 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:27.862798 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856bc659-07ba-4637-9207-164c347c2c04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:27.863850 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856bc659-07ba-4637-9207-164c347c2c04 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:27.864311 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856bc659-07ba-4637-9207-164c347c2c04 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:27.865227 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856bc659-07ba-4637-9207-164c347c2c04 None admin] PATH_INFO: `/v3/domains` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:27.866306 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856bc659-07ba-4637-9207-164c347c2c04 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:59:27.866776 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856bc659-07ba-4637-9207-164c347c2c04 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:27.869507 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856bc659-07ba-4637-9207-164c347c2c04 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:27.885753 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 313/653] 10.222.0.55 () {56 vars in 1109 bytes} [Fri Sep 23 14:59:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:59:27.891297 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856bc659-07ba-4637-9207-164c347c2c04 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:27.894226 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:27.895662 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 341/654] 10.222.0.55 () {58 vars in 1282 bytes} [Fri Sep 23 14:59:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 23 14:59:27.899437 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19341af-ccba-49ef-8314-303906f0bc49 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:28.027624 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19341af-ccba-49ef-8314-303906f0bc49 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:28.034221 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856bc659-07ba-4637-9207-164c347c2c04 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:28.069608 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c91298f-392d-49fb-bd5c-98bf6926ae25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:28.070279 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c91298f-392d-49fb-bd5c-98bf6926ae25 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:28.070566 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c91298f-392d-49fb-bd5c-98bf6926ae25 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:28.070842 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c91298f-392d-49fb-bd5c-98bf6926ae25 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:28.073824 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b9d6e9-143b-4803-ab8c-0bb084364f74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:28.074437 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b9d6e9-143b-4803-ab8c-0bb084364f74 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:28.074716 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b9d6e9-143b-4803-ab8c-0bb084364f74 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:28.074990 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b9d6e9-143b-4803-ab8c-0bb084364f74 None admin] PATH_INFO: `/v3/domains` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:28.075659 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b9d6e9-143b-4803-ab8c-0bb084364f74 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:59:28.075967 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b9d6e9-143b-4803-ab8c-0bb084364f74 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:28.079169 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b9d6e9-143b-4803-ab8c-0bb084364f74 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:28.079294 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c91298f-392d-49fb-bd5c-98bf6926ae25 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:28.080443 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c91298f-392d-49fb-bd5c-98bf6926ae25 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:28.092107 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 314/655] 10.222.0.55 () {56 vars in 1109 bytes} [Fri Sep 23 14:59:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:59:28.092107 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b9d6e9-143b-4803-ab8c-0bb084364f74 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:28.093539 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-5c91298f-392d-49fb-bd5c-98bf6926ae25 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 23 14:59:28.093539 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 14:59:28.093539 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 14:59:28.093539 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 14:59:28.093539 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 14:59:28.093539 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 14:59:28.093539 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 14:59:28.093539 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 14:59:28.093539 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 14:59:28.093539 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 14:59:28.093539 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 14:59:28.093539 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 14:59:28.093539 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 23 14:59:28.093539 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 23 14:59:28.093539 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 23 14:59:28.093539 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 23 14:59:28.094448 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 14:59:28.094448 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 14:59:28.094448 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 23 14:59:28.094448 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 14:59:28.094448 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 23 14:59:28.094448 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 14:59:28.094448 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 14:59:28.094448 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 14:59:28.094448 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 14:59:28.094448 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 14:59:28.094448 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 14:59:28.094448 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Sep 23 14:59:28.094448 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 14:59:28.094448 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Sep 23 14:59:28.094448 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 14:59:28.094448 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 14:59:28.094448 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 14:59:28.094448 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Sep 23 14:59:28.095439 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 14:59:28.095439 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Sep 23 14:59:28.095439 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 23 14:59:28.095439 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 23 14:59:28.095439 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 23 14:59:28.095439 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 23 14:59:28.095439 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 23 14:59:28.095439 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 23 14:59:28.095439 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 23 14:59:28.095439 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Sep 23 14:59:28.099405 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 342/656] 10.222.0.55 () {56 vars in 1099 bytes} [Fri Sep 23 14:59:27 2022] GET /identity/v3/users/cinder => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 14:59:28.101593 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c5be10-ba02-4b46-9a24-e6d7f8e409f7 None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:28.101885 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c5be10-ba02-4b46-9a24-e6d7f8e409f7 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:28.102162 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c5be10-ba02-4b46-9a24-e6d7f8e409f7 None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:28.108139 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 343/657] 10.222.0.55 () {54 vars in 927 bytes} [Fri Sep 23 14:59:28 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 14:59:28.108139 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c5be10-ba02-4b46-9a24-e6d7f8e409f7 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:28.126234 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:28.162952 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:28.203516 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b9d6e9-143b-4803-ab8c-0bb084364f74 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:28.224080 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c5be10-ba02-4b46-9a24-e6d7f8e409f7 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:28.242403 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e440c53-5626-4e20-a045-9e6153c514fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:28.243041 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e440c53-5626-4e20-a045-9e6153c514fc None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:28.243347 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e440c53-5626-4e20-a045-9e6153c514fc None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:28.243617 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e440c53-5626-4e20-a045-9e6153c514fc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:28.254271 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e440c53-5626-4e20-a045-9e6153c514fc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:28.255720 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e440c53-5626-4e20-a045-9e6153c514fc None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:28.266373 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b907d1a4-8f4d-4ab0-8fb6-105b9470fa35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:28.266995 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b907d1a4-8f4d-4ab0-8fb6-105b9470fa35 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:28.267282 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b907d1a4-8f4d-4ab0-8fb6-105b9470fa35 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:28.267555 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b907d1a4-8f4d-4ab0-8fb6-105b9470fa35 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:28.270944 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application [None req-6e440c53-5626-4e20-a045-9e6153c514fc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 14:59:28.270944 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 14:59:28.270944 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 14:59:28.270944 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 14:59:28.270944 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 14:59:28.270944 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 14:59:28.270944 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 14:59:28.270944 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 14:59:28.270944 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 14:59:28.270944 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 14:59:28.270944 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 14:59:28.270944 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 14:59:28.270944 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 14:59:28.270944 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 14:59:28.270944 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 14:59:28.270944 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 14:59:28.271888 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 14:59:28.271888 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 14:59:28.271888 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 14:59:28.271888 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 14:59:28.271888 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 14:59:28.271888 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 14:59:28.271888 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 14:59:28.271888 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application with Lock( Sep 23 14:59:28.271888 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 14:59:28.271888 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._enter() Sep 23 14:59:28.271888 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 14:59:28.271888 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 14:59:28.271888 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 14:59:28.271888 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.creator() Sep 23 14:59:28.271888 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 14:59:28.271888 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application created_value = creator( Sep 23 14:59:28.271888 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 14:59:28.272814 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 14:59:28.272814 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 14:59:28.272814 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Sep 23 14:59:28.275900 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b907d1a4-8f4d-4ab0-8fb6-105b9470fa35 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:28.277230 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 315/658] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Sep 23 14:59:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 14:59:28.277541 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b907d1a4-8f4d-4ab0-8fb6-105b9470fa35 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:28.278089 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e440c53-5626-4e20-a045-9e6153c514fc None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:28.292415 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-b907d1a4-8f4d-4ab0-8fb6-105b9470fa35 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 23 14:59:28.292415 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 14:59:28.292415 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 14:59:28.292415 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 14:59:28.292415 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 14:59:28.292415 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 14:59:28.292415 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 14:59:28.292415 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 14:59:28.292415 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 14:59:28.292415 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 14:59:28.292415 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 14:59:28.292415 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 14:59:28.292415 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 23 14:59:28.292415 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 23 14:59:28.292415 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 23 14:59:28.292415 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 23 14:59:28.293599 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 14:59:28.293599 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 14:59:28.293599 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 23 14:59:28.293599 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 14:59:28.293599 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 23 14:59:28.293599 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 14:59:28.293599 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 14:59:28.293599 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 14:59:28.293599 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 14:59:28.293599 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 14:59:28.293599 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 14:59:28.293599 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Sep 23 14:59:28.293599 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 14:59:28.293599 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Sep 23 14:59:28.293599 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 14:59:28.293599 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 14:59:28.293599 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 14:59:28.293599 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Sep 23 14:59:28.294724 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 14:59:28.294724 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Sep 23 14:59:28.294724 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 23 14:59:28.294724 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 23 14:59:28.294724 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 23 14:59:28.294724 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 23 14:59:28.294724 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 23 14:59:28.294724 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 23 14:59:28.294724 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 23 14:59:28.294724 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Sep 23 14:59:28.294724 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 344/659] 10.222.0.55 () {56 vars in 1102 bytes} [Fri Sep 23 14:59:28 2022] GET /identity/v3/users/neutron => generated 85 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 14:59:28.299489 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad56ae4-1795-4ab8-86b0-f902efc4a888 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:28.299772 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad56ae4-1795-4ab8-86b0-f902efc4a888 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:28.300068 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad56ae4-1795-4ab8-86b0-f902efc4a888 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:28.318396 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-0ad56ae4-1795-4ab8-86b0-f902efc4a888 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 14:59:28.319592 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-0ad56ae4-1795-4ab8-86b0-f902efc4a888 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 14:59:28.388258 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e440c53-5626-4e20-a045-9e6153c514fc None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:28.394127 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 345/660] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Sep 23 14:59:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 14:59:28.394817 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad56ae4-1795-4ab8-86b0-f902efc4a888 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:28.397200 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:28.414902 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:28.440696 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c398dd06-84b1-4780-97aa-db9567c19e27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:28.442304 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c398dd06-84b1-4780-97aa-db9567c19e27 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:28.442794 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c398dd06-84b1-4780-97aa-db9567c19e27 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:28.443619 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c398dd06-84b1-4780-97aa-db9567c19e27 None admin] PATH_INFO: `/v3/users` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:28.444640 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c398dd06-84b1-4780-97aa-db9567c19e27 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:59:28.445600 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c398dd06-84b1-4780-97aa-db9567c19e27 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:28.448725 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c398dd06-84b1-4780-97aa-db9567c19e27 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:28.473193 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 316/661] 10.222.0.55 () {56 vars in 1137 bytes} [Fri Sep 23 14:59:28 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:59:28.474411 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c398dd06-84b1-4780-97aa-db9567c19e27 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:28.514423 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad56ae4-1795-4ab8-86b0-f902efc4a888 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:28.558539 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3f5a3f-5b46-4a00-b9e5-8ac8b9a8eff2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:28.559156 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3f5a3f-5b46-4a00-b9e5-8ac8b9a8eff2 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:28.559439 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3f5a3f-5b46-4a00-b9e5-8ac8b9a8eff2 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:28.559715 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3f5a3f-5b46-4a00-b9e5-8ac8b9a8eff2 None admin] PATH_INFO: `/v3/domains` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:28.560391 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3f5a3f-5b46-4a00-b9e5-8ac8b9a8eff2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:59:28.560691 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3f5a3f-5b46-4a00-b9e5-8ac8b9a8eff2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:28.562140 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3f5a3f-5b46-4a00-b9e5-8ac8b9a8eff2 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:28.579000 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 346/662] 10.222.0.55 () {56 vars in 1109 bytes} [Fri Sep 23 14:59:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:59:28.579000 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3f5a3f-5b46-4a00-b9e5-8ac8b9a8eff2 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:28.623516 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c398dd06-84b1-4780-97aa-db9567c19e27 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:28.645761 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:28.671999 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b781cd55-230b-410e-94fd-e2e71a648965 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:28.672619 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b781cd55-230b-410e-94fd-e2e71a648965 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:28.672900 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b781cd55-230b-410e-94fd-e2e71a648965 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:28.673175 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b781cd55-230b-410e-94fd-e2e71a648965 None admin] PATH_INFO: `/v3/users` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:28.673909 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b781cd55-230b-410e-94fd-e2e71a648965 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:59:28.674214 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b781cd55-230b-410e-94fd-e2e71a648965 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:28.675641 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b781cd55-230b-410e-94fd-e2e71a648965 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:28.686855 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3f5a3f-5b46-4a00-b9e5-8ac8b9a8eff2 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:28.693717 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 317/663] 10.222.0.55 () {56 vars in 1139 bytes} [Fri Sep 23 14:59:28 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:59:28.694537 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b781cd55-230b-410e-94fd-e2e71a648965 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:28.724347 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c23bb2b-42b1-4a52-a509-aed41ba4e305 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:28.725297 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c23bb2b-42b1-4a52-a509-aed41ba4e305 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:28.725777 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c23bb2b-42b1-4a52-a509-aed41ba4e305 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:28.726227 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c23bb2b-42b1-4a52-a509-aed41ba4e305 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:28.726801 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:28.799916 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b781cd55-230b-410e-94fd-e2e71a648965 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:28.804069 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c23bb2b-42b1-4a52-a509-aed41ba4e305 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:28.805634 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c23bb2b-42b1-4a52-a509-aed41ba4e305 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:28.845784 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8b0f5f-df3f-4804-b453-f5d46efe6e54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:28.847191 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8b0f5f-df3f-4804-b453-f5d46efe6e54 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:28.847679 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8b0f5f-df3f-4804-b453-f5d46efe6e54 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:28.848628 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8b0f5f-df3f-4804-b453-f5d46efe6e54 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:28.858684 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8b0f5f-df3f-4804-b453-f5d46efe6e54 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:28.860199 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8b0f5f-df3f-4804-b453-f5d46efe6e54 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:28.870684 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:28.873288 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application [None req-af8b0f5f-df3f-4804-b453-f5d46efe6e54 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 14:59:28.873288 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 14:59:28.873288 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 14:59:28.873288 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 14:59:28.873288 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 14:59:28.873288 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 14:59:28.873288 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 14:59:28.873288 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 14:59:28.873288 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 14:59:28.873288 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 14:59:28.873288 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 14:59:28.873288 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 14:59:28.873288 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 23 14:59:28.873288 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 23 14:59:28.873288 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 23 14:59:28.874503 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 23 14:59:28.874503 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 14:59:28.874503 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 14:59:28.874503 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 14:59:28.874503 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 14:59:28.874503 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 14:59:28.874503 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 14:59:28.874503 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 14:59:28.874503 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application with Lock( Sep 23 14:59:28.874503 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 14:59:28.874503 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._enter() Sep 23 14:59:28.874503 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 14:59:28.874503 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 14:59:28.874503 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 14:59:28.874503 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.creator() Sep 23 14:59:28.874503 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 14:59:28.874503 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application created_value = creator( Sep 23 14:59:28.875475 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 23 14:59:28.875475 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 23 14:59:28.875475 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 23 14:59:28.875475 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 23 14:59:28.875475 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 23 14:59:28.875475 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 23 14:59:28.875475 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 14:59:28.875475 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Sep 23 14:59:28.875475 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 318/664] 10.222.0.55 () {56 vars in 1111 bytes} [Fri Sep 23 14:59:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 14:59:28.875475 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8b0f5f-df3f-4804-b453-f5d46efe6e54 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:28.921103 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 347/665] 10.222.0.55 () {58 vars in 1282 bytes} [Fri Sep 23 14:59:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 23 14:59:28.922229 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c23bb2b-42b1-4a52-a509-aed41ba4e305 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:28.974385 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:28.984080 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8b0f5f-df3f-4804-b453-f5d46efe6e54 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:29.022218 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f816718-2805-48e2-88b2-91fe61e462bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:29.022848 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f816718-2805-48e2-88b2-91fe61e462bf None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:29.023125 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f816718-2805-48e2-88b2-91fe61e462bf None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:29.023391 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f816718-2805-48e2-88b2-91fe61e462bf None admin] PATH_INFO: `/v3/roles/service` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:29.034637 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f816718-2805-48e2-88b2-91fe61e462bf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 14:59:29.037037 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c23bb2b-42b1-4a52-a509-aed41ba4e305 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:29.039419 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application [None req-5f816718-2805-48e2-88b2-91fe61e462bf None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 23 14:59:29.039419 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 14:59:29.039419 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 14:59:29.039419 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 14:59:29.039419 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 14:59:29.039419 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 14:59:29.039419 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 14:59:29.039419 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 14:59:29.039419 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 14:59:29.039419 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 14:59:29.039419 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 14:59:29.039419 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 14:59:29.039419 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 14:59:29.039419 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 14:59:29.039419 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 14:59:29.039419 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 14:59:29.040566 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 14:59:29.040566 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 14:59:29.040566 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 14:59:29.040566 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 14:59:29.040566 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 14:59:29.040566 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 14:59:29.040566 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 14:59:29.040566 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 14:59:29.040566 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 14:59:29.040566 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 14:59:29.040566 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 14:59:29.040566 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application with Lock( Sep 23 14:59:29.040566 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 14:59:29.040566 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._enter() Sep 23 14:59:29.040566 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 14:59:29.040566 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 14:59:29.041412 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 14:59:29.041412 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.creator() Sep 23 14:59:29.041412 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 14:59:29.041412 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application created_value = creator( Sep 23 14:59:29.041412 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 14:59:29.041412 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 14:59:29.041412 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 14:59:29.041412 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 14:59:29.041412 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 14:59:29.041412 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 14:59:29.041412 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 23 14:59:29.041412 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Sep 23 14:59:29.041412 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 319/666] 10.222.0.55 () {56 vars in 1102 bytes} [Fri Sep 23 14:59:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 14:59:29.041412 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f816718-2805-48e2-88b2-91fe61e462bf None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:29.080825 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e373c6a3-f087-40c3-96ec-cd33c432eb03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:29.081453 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e373c6a3-f087-40c3-96ec-cd33c432eb03 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:29.081800 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e373c6a3-f087-40c3-96ec-cd33c432eb03 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:29.082147 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e373c6a3-f087-40c3-96ec-cd33c432eb03 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:29.117780 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:29.158690 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f816718-2805-48e2-88b2-91fe61e462bf None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:29.159700 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e373c6a3-f087-40c3-96ec-cd33c432eb03 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:29.161184 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e373c6a3-f087-40c3-96ec-cd33c432eb03 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:29.193426 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191817f1-2fac-4cbc-b97b-37986e5e0eed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:29.194052 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191817f1-2fac-4cbc-b97b-37986e5e0eed None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:29.194331 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191817f1-2fac-4cbc-b97b-37986e5e0eed None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:29.194604 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191817f1-2fac-4cbc-b97b-37986e5e0eed None admin] PATH_INFO: `/v3/projects` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:29.195287 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191817f1-2fac-4cbc-b97b-37986e5e0eed None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:59:29.195587 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191817f1-2fac-4cbc-b97b-37986e5e0eed None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:29.197688 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191817f1-2fac-4cbc-b97b-37986e5e0eed None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:29.214503 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 320/667] 10.222.0.55 () {56 vars in 1148 bytes} [Fri Sep 23 14:59:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:59:29.215352 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191817f1-2fac-4cbc-b97b-37986e5e0eed None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:29.229329 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:29.288350 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 348/668] 10.222.0.55 () {58 vars in 1282 bytes} [Fri Sep 23 14:59:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 23 14:59:29.288350 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e373c6a3-f087-40c3-96ec-cd33c432eb03 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:29.328311 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191817f1-2fac-4cbc-b97b-37986e5e0eed None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:29.356759 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:29.369780 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5084d90c-0603-4344-b818-ad99402911f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:29.370668 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5084d90c-0603-4344-b818-ad99402911f1 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:29.371077 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5084d90c-0603-4344-b818-ad99402911f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:29.371352 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5084d90c-0603-4344-b818-ad99402911f1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:29.382774 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5084d90c-0603-4344-b818-ad99402911f1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:29.384378 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5084d90c-0603-4344-b818-ad99402911f1 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:29.397765 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e373c6a3-f087-40c3-96ec-cd33c432eb03 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:29.398772 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application [None req-5084d90c-0603-4344-b818-ad99402911f1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 14:59:29.398772 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 14:59:29.398772 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 14:59:29.398772 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 14:59:29.398772 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 14:59:29.398772 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 14:59:29.398772 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 14:59:29.398772 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 14:59:29.398772 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 14:59:29.398772 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 14:59:29.398772 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 14:59:29.398772 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 14:59:29.398772 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 14:59:29.398772 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 14:59:29.398772 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 14:59:29.398772 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 14:59:29.399698 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 14:59:29.399698 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 14:59:29.399698 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 14:59:29.399698 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 14:59:29.399698 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 14:59:29.399698 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 14:59:29.399698 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 14:59:29.399698 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application with Lock( Sep 23 14:59:29.399698 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 14:59:29.399698 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._enter() Sep 23 14:59:29.399698 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 14:59:29.399698 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 14:59:29.399698 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 14:59:29.399698 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.creator() Sep 23 14:59:29.399698 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 14:59:29.399698 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application created_value = creator( Sep 23 14:59:29.399698 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 14:59:29.400636 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 14:59:29.400636 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 14:59:29.400636 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Sep 23 14:59:29.400636 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 321/669] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Sep 23 14:59:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 14:59:29.401458 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5084d90c-0603-4344-b818-ad99402911f1 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:29.440449 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df7f885-6b39-4e3e-ad36-65ea9d301c58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:29.441112 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df7f885-6b39-4e3e-ad36-65ea9d301c58 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:29.441404 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df7f885-6b39-4e3e-ad36-65ea9d301c58 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:29.441678 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df7f885-6b39-4e3e-ad36-65ea9d301c58 None admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:29.442343 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df7f885-6b39-4e3e-ad36-65ea9d301c58 None admin] RBAC: Adding query filter params (name=service) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:59:29.442647 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df7f885-6b39-4e3e-ad36-65ea9d301c58 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:29.444078 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df7f885-6b39-4e3e-ad36-65ea9d301c58 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:29.452978 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 349/670] 10.222.0.55 () {56 vars in 1103 bytes} [Fri Sep 23 14:59:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:59:29.453799 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df7f885-6b39-4e3e-ad36-65ea9d301c58 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:29.472606 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:29.515443 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5084d90c-0603-4344-b818-ad99402911f1 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:29.553148 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9caa7c-5e87-4889-958d-9f214153d5be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:29.553809 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9caa7c-5e87-4889-958d-9f214153d5be None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:29.554102 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9caa7c-5e87-4889-958d-9f214153d5be None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:29.554424 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9caa7c-5e87-4889-958d-9f214153d5be None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:29.555156 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9caa7c-5e87-4889-958d-9f214153d5be None admin] RBAC: Adding query filter params (role.id=0665869b71fa40b0b552efdd57faf023, scope.project.id=87b3e9d4b6474bd4a04ba59ef52bf3e2, user.id=51c39f1d05634589a88c0b6477bd014e) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:59:29.555472 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9caa7c-5e87-4889-958d-9f214153d5be None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:29.557259 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9caa7c-5e87-4889-958d-9f214153d5be None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:29.561123 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df7f885-6b39-4e3e-ad36-65ea9d301c58 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:29.563701 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 322/671] 10.222.0.55 () {56 vars in 1374 bytes} [Fri Sep 23 14:59:29 2022] GET /identity/v3/role_assignments?user.id=51c39f1d05634589a88c0b6477bd014e&scope.project.id=87b3e9d4b6474bd4a04ba59ef52bf3e2&role.id=0665869b71fa40b0b552efdd57faf023 => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:59:29.564612 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9caa7c-5e87-4889-958d-9f214153d5be None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:29.602352 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:29.617350 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ea6cd5-40d4-4015-bc5d-2749afbe6a21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:29.618249 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ea6cd5-40d4-4015-bc5d-2749afbe6a21 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:29.618687 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ea6cd5-40d4-4015-bc5d-2749afbe6a21 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:29.619079 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ea6cd5-40d4-4015-bc5d-2749afbe6a21 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:29.632666 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ea6cd5-40d4-4015-bc5d-2749afbe6a21 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:29.634995 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ea6cd5-40d4-4015-bc5d-2749afbe6a21 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:29.653194 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-89ea6cd5-40d4-4015-bc5d-2749afbe6a21 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 14:59:29.653194 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 14:59:29.653194 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 14:59:29.653194 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 14:59:29.653194 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 14:59:29.653194 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 14:59:29.653194 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 14:59:29.653194 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 14:59:29.653194 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 14:59:29.653194 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 14:59:29.653194 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 14:59:29.653194 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 14:59:29.653194 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 14:59:29.653194 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 14:59:29.653194 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 14:59:29.653194 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 14:59:29.654381 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 14:59:29.654381 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 14:59:29.654381 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 14:59:29.654381 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 14:59:29.654381 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 14:59:29.654381 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 14:59:29.654381 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 14:59:29.654381 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Sep 23 14:59:29.654381 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 14:59:29.654381 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Sep 23 14:59:29.654381 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 14:59:29.654381 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 14:59:29.654381 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 14:59:29.654381 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Sep 23 14:59:29.654381 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 14:59:29.654381 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Sep 23 14:59:29.654381 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 14:59:29.655291 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 14:59:29.655291 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 14:59:29.655291 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Sep 23 14:59:29.662189 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 350/672] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Sep 23 14:59:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 14:59:29.663566 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ea6cd5-40d4-4015-bc5d-2749afbe6a21 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:29.704788 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9caa7c-5e87-4889-958d-9f214153d5be None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:29.740969 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:29.757529 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516682b2-6651-4845-903a-f480bb938f29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:29.758153 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516682b2-6651-4845-903a-f480bb938f29 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:29.758434 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516682b2-6651-4845-903a-f480bb938f29 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:29.758708 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516682b2-6651-4845-903a-f480bb938f29 None admin] PATH_INFO: `/v3/domains` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:29.761215 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516682b2-6651-4845-903a-f480bb938f29 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:59:29.761529 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516682b2-6651-4845-903a-f480bb938f29 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:29.763186 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516682b2-6651-4845-903a-f480bb938f29 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:29.778700 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 323/673] 10.222.0.55 () {56 vars in 1109 bytes} [Fri Sep 23 14:59:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:59:29.779958 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516682b2-6651-4845-903a-f480bb938f29 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:29.793842 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ea6cd5-40d4-4015-bc5d-2749afbe6a21 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:29.842207 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e44888-73aa-4ff4-a075-8ad92b62d1bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:29.843399 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e44888-73aa-4ff4-a075-8ad92b62d1bb None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:29.843898 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e44888-73aa-4ff4-a075-8ad92b62d1bb None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:29.844487 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e44888-73aa-4ff4-a075-8ad92b62d1bb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:29.859548 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e44888-73aa-4ff4-a075-8ad92b62d1bb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:29.861952 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e44888-73aa-4ff4-a075-8ad92b62d1bb None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:29.879542 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-21e44888-73aa-4ff4-a075-8ad92b62d1bb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 14:59:29.879542 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 14:59:29.879542 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 14:59:29.879542 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 14:59:29.879542 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 14:59:29.879542 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 14:59:29.879542 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 14:59:29.879542 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 14:59:29.879542 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 14:59:29.879542 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 14:59:29.879542 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 14:59:29.879542 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 14:59:29.879542 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 14:59:29.879542 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 14:59:29.879542 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 14:59:29.879542 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 14:59:29.880658 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 14:59:29.880658 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 14:59:29.880658 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 14:59:29.880658 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 14:59:29.880658 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 14:59:29.880658 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 14:59:29.880658 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 14:59:29.880658 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Sep 23 14:59:29.880658 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 14:59:29.880658 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Sep 23 14:59:29.880658 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 14:59:29.880658 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 14:59:29.880658 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 14:59:29.880658 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Sep 23 14:59:29.880658 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 14:59:29.880658 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Sep 23 14:59:29.880658 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 14:59:29.881550 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 14:59:29.881550 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 14:59:29.881550 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Sep 23 14:59:29.882435 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 351/674] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Sep 23 14:59:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 14:59:29.883174 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e44888-73aa-4ff4-a075-8ad92b62d1bb None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:29.916716 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516682b2-6651-4845-903a-f480bb938f29 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:29.918225 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:29.957238 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55db6888-637c-47b7-94a4-e4b91f40b428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:29.957764 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55db6888-637c-47b7-94a4-e4b91f40b428 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:29.958044 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55db6888-637c-47b7-94a4-e4b91f40b428 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:29.958550 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55db6888-637c-47b7-94a4-e4b91f40b428 None admin] PATH_INFO: `/v3/domains` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:29.958952 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55db6888-637c-47b7-94a4-e4b91f40b428 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:59:29.959253 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55db6888-637c-47b7-94a4-e4b91f40b428 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:29.960729 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55db6888-637c-47b7-94a4-e4b91f40b428 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:29.977024 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 324/675] 10.222.0.55 () {56 vars in 1109 bytes} [Fri Sep 23 14:59:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:59:29.977024 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55db6888-637c-47b7-94a4-e4b91f40b428 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:30.009501 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e44888-73aa-4ff4-a075-8ad92b62d1bb None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:30.026609 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:30.046617 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6aa9742-477b-488f-b899-fa4cece7f893 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:30.047405 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6aa9742-477b-488f-b899-fa4cece7f893 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:30.047694 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6aa9742-477b-488f-b899-fa4cece7f893 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:30.047970 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6aa9742-477b-488f-b899-fa4cece7f893 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:30.054931 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6aa9742-477b-488f-b899-fa4cece7f893 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:30.056991 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6aa9742-477b-488f-b899-fa4cece7f893 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:30.069250 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-d6aa9742-477b-488f-b899-fa4cece7f893 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 14:59:30.069250 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 14:59:30.069250 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 14:59:30.069250 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 14:59:30.069250 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 14:59:30.069250 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 14:59:30.069250 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 14:59:30.069250 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 14:59:30.069250 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 14:59:30.069250 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 14:59:30.069250 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 14:59:30.069250 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 14:59:30.069250 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 23 14:59:30.069250 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 23 14:59:30.069250 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 23 14:59:30.070243 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 23 14:59:30.070243 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 14:59:30.070243 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 14:59:30.070243 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 14:59:30.070243 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 14:59:30.070243 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 14:59:30.070243 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 14:59:30.070243 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 14:59:30.070243 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Sep 23 14:59:30.070243 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 14:59:30.070243 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Sep 23 14:59:30.070243 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 14:59:30.070243 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 14:59:30.070243 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 14:59:30.070243 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Sep 23 14:59:30.070243 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 14:59:30.070243 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Sep 23 14:59:30.071144 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 23 14:59:30.071144 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 23 14:59:30.071144 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 23 14:59:30.071144 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 23 14:59:30.071144 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 23 14:59:30.071144 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 23 14:59:30.071144 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 14:59:30.071144 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Sep 23 14:59:30.071144 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 352/676] 10.222.0.55 () {56 vars in 1111 bytes} [Fri Sep 23 14:59:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 14:59:30.072766 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec6a74a-7b45-4979-961b-a9478e4fdb30 None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:30.073059 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec6a74a-7b45-4979-961b-a9478e4fdb30 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:30.073638 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec6a74a-7b45-4979-961b-a9478e4fdb30 None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:30.081875 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 353/677] 10.222.0.55 () {54 vars in 927 bytes} [Fri Sep 23 14:59:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 14:59:30.081875 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec6a74a-7b45-4979-961b-a9478e4fdb30 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:30.100316 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55db6888-637c-47b7-94a4-e4b91f40b428 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:30.138227 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81078dd-aa33-4c3f-bbe3-7bd034218717 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:30.138961 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81078dd-aa33-4c3f-bbe3-7bd034218717 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:30.139326 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81078dd-aa33-4c3f-bbe3-7bd034218717 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:30.139670 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81078dd-aa33-4c3f-bbe3-7bd034218717 None admin] PATH_INFO: `/v3/domains` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:30.140443 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81078dd-aa33-4c3f-bbe3-7bd034218717 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:59:30.140782 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81078dd-aa33-4c3f-bbe3-7bd034218717 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:30.142221 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81078dd-aa33-4c3f-bbe3-7bd034218717 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:30.162807 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 325/678] 10.222.0.55 () {56 vars in 1109 bytes} [Fri Sep 23 14:59:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:59:30.163839 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81078dd-aa33-4c3f-bbe3-7bd034218717 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:30.185068 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:30.202366 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec6a74a-7b45-4979-961b-a9478e4fdb30 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:30.240415 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8bdb17-beac-4c04-8b09-136888e81221 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:30.241194 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8bdb17-beac-4c04-8b09-136888e81221 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:30.241475 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8bdb17-beac-4c04-8b09-136888e81221 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:30.241757 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8bdb17-beac-4c04-8b09-136888e81221 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:30.249500 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8bdb17-beac-4c04-8b09-136888e81221 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:30.251381 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8bdb17-beac-4c04-8b09-136888e81221 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:30.262941 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-1d8bdb17-beac-4c04-8b09-136888e81221 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 14:59:30.262941 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 14:59:30.262941 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 14:59:30.262941 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 14:59:30.262941 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 14:59:30.262941 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 14:59:30.262941 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 14:59:30.262941 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 14:59:30.262941 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 14:59:30.262941 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 14:59:30.262941 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 14:59:30.262941 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 14:59:30.262941 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 23 14:59:30.262941 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 23 14:59:30.262941 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 23 14:59:30.263890 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 23 14:59:30.263890 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 14:59:30.263890 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 14:59:30.263890 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 14:59:30.263890 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 14:59:30.263890 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 14:59:30.263890 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 14:59:30.263890 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 14:59:30.263890 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Sep 23 14:59:30.263890 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 14:59:30.263890 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Sep 23 14:59:30.263890 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 14:59:30.263890 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 14:59:30.263890 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 14:59:30.263890 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Sep 23 14:59:30.263890 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 14:59:30.263890 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Sep 23 14:59:30.264818 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 23 14:59:30.264818 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 23 14:59:30.264818 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 23 14:59:30.264818 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 23 14:59:30.264818 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 23 14:59:30.264818 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 23 14:59:30.264818 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 14:59:30.264818 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Sep 23 14:59:30.269036 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 354/679] 10.222.0.55 () {56 vars in 1111 bytes} [Fri Sep 23 14:59:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 14:59:30.269979 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2239dc-d6b0-4bdc-aca2-487c24a13812 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:30.270274 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2239dc-d6b0-4bdc-aca2-487c24a13812 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:30.270556 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2239dc-d6b0-4bdc-aca2-487c24a13812 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:30.276586 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81078dd-aa33-4c3f-bbe3-7bd034218717 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:30.285733 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:30.290489 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-ef2239dc-d6b0-4bdc-aca2-487c24a13812 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 14:59:30.292076 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-ef2239dc-d6b0-4bdc-aca2-487c24a13812 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 14:59:30.317455 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d7f63b-93b0-4650-88ed-090f119f2165 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:30.318068 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d7f63b-93b0-4650-88ed-090f119f2165 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:30.318349 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d7f63b-93b0-4650-88ed-090f119f2165 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:30.318624 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d7f63b-93b0-4650-88ed-090f119f2165 None admin] PATH_INFO: `/v3/projects` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:30.319316 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d7f63b-93b0-4650-88ed-090f119f2165 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:59:30.319625 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d7f63b-93b0-4650-88ed-090f119f2165 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:30.321092 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d7f63b-93b0-4650-88ed-090f119f2165 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:30.339954 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 326/680] 10.222.0.55 () {56 vars in 1148 bytes} [Fri Sep 23 14:59:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:59:30.339954 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d7f63b-93b0-4650-88ed-090f119f2165 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:30.380003 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 355/681] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Sep 23 14:59:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 14:59:30.380003 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2239dc-d6b0-4bdc-aca2-487c24a13812 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:30.448744 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d7f63b-93b0-4650-88ed-090f119f2165 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:30.485207 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:30.488444 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324bf035-3842-402c-a4ce-0ed61773c508 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:30.489290 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324bf035-3842-402c-a4ce-0ed61773c508 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:30.489574 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324bf035-3842-402c-a4ce-0ed61773c508 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:30.489847 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324bf035-3842-402c-a4ce-0ed61773c508 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:30.498099 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324bf035-3842-402c-a4ce-0ed61773c508 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:30.499526 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324bf035-3842-402c-a4ce-0ed61773c508 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:30.513231 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application [None req-324bf035-3842-402c-a4ce-0ed61773c508 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 23 14:59:30.513231 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 14:59:30.513231 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 14:59:30.513231 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 14:59:30.513231 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 14:59:30.513231 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 14:59:30.513231 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 14:59:30.513231 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 14:59:30.513231 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 14:59:30.513231 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 14:59:30.513231 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 14:59:30.513231 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 14:59:30.513231 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 23 14:59:30.513231 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 23 14:59:30.513231 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 23 14:59:30.513231 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 23 14:59:30.514331 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 14:59:30.514331 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 14:59:30.514331 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 23 14:59:30.514331 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 14:59:30.514331 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 23 14:59:30.514331 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 14:59:30.514331 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 14:59:30.514331 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 14:59:30.514331 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 14:59:30.514331 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 14:59:30.514331 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 14:59:30.514331 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application with Lock( Sep 23 14:59:30.514331 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 14:59:30.514331 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._enter() Sep 23 14:59:30.514331 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 14:59:30.514331 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 14:59:30.514331 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 14:59:30.514331 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.creator() Sep 23 14:59:30.515358 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 14:59:30.515358 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application created_value = creator( Sep 23 14:59:30.515358 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 23 14:59:30.515358 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 23 14:59:30.515358 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 23 14:59:30.515358 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 23 14:59:30.515358 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 23 14:59:30.515358 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 23 14:59:30.515358 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 23 14:59:30.515358 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Sep 23 14:59:30.516926 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2239dc-d6b0-4bdc-aca2-487c24a13812 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:30.519142 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 327/682] 10.222.0.55 () {56 vars in 1096 bytes} [Fri Sep 23 14:59:30 2022] GET /identity/v3/users/swift => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 14:59:30.520013 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324bf035-3842-402c-a4ce-0ed61773c508 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:30.554490 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:30.568590 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649ddc3b-b68c-4a66-bc6e-02f3443e9858 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:30.569605 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649ddc3b-b68c-4a66-bc6e-02f3443e9858 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:30.570484 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649ddc3b-b68c-4a66-bc6e-02f3443e9858 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:30.570975 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649ddc3b-b68c-4a66-bc6e-02f3443e9858 None admin] PATH_INFO: `/v3/projects` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:30.572433 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649ddc3b-b68c-4a66-bc6e-02f3443e9858 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:59:30.572950 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649ddc3b-b68c-4a66-bc6e-02f3443e9858 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:30.575283 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649ddc3b-b68c-4a66-bc6e-02f3443e9858 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:30.594428 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 356/683] 10.222.0.55 () {56 vars in 1148 bytes} [Fri Sep 23 14:59:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:59:30.595694 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649ddc3b-b68c-4a66-bc6e-02f3443e9858 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:30.644258 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324bf035-3842-402c-a4ce-0ed61773c508 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:30.684716 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a578fce5-cdf2-4d6d-8b16-f3a3e2dc502b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:30.685331 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a578fce5-cdf2-4d6d-8b16-f3a3e2dc502b None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:30.685611 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a578fce5-cdf2-4d6d-8b16-f3a3e2dc502b None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:30.685883 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a578fce5-cdf2-4d6d-8b16-f3a3e2dc502b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:30.686604 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a578fce5-cdf2-4d6d-8b16-f3a3e2dc502b None admin] RBAC: Adding query filter params (role.id=0665869b71fa40b0b552efdd57faf023, scope.project.id=87b3e9d4b6474bd4a04ba59ef52bf3e2, user.id=feca6278be904ef8b0dc580a67e477f8) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:59:30.686930 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a578fce5-cdf2-4d6d-8b16-f3a3e2dc502b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:30.688373 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a578fce5-cdf2-4d6d-8b16-f3a3e2dc502b None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:30.694316 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 328/684] 10.222.0.55 () {56 vars in 1374 bytes} [Fri Sep 23 14:59:30 2022] GET /identity/v3/role_assignments?user.id=feca6278be904ef8b0dc580a67e477f8&scope.project.id=87b3e9d4b6474bd4a04ba59ef52bf3e2&role.id=0665869b71fa40b0b552efdd57faf023 => generated 608 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:59:30.698930 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a578fce5-cdf2-4d6d-8b16-f3a3e2dc502b None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:30.732238 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649ddc3b-b68c-4a66-bc6e-02f3443e9858 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:30.779556 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07985c1d-a7e3-496e-a74b-1675392d5129 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:30.780379 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07985c1d-a7e3-496e-a74b-1675392d5129 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:30.780732 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07985c1d-a7e3-496e-a74b-1675392d5129 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:30.781078 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07985c1d-a7e3-496e-a74b-1675392d5129 None admin] PATH_INFO: `/v3/users` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:30.781900 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07985c1d-a7e3-496e-a74b-1675392d5129 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:59:30.782279 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07985c1d-a7e3-496e-a74b-1675392d5129 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:30.784315 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07985c1d-a7e3-496e-a74b-1675392d5129 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:30.795814 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:30.799721 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:30.811027 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 357/685] 10.222.0.55 () {56 vars in 1135 bytes} [Fri Sep 23 14:59:30 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:59:30.811939 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07985c1d-a7e3-496e-a74b-1675392d5129 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:30.812696 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a578fce5-cdf2-4d6d-8b16-f3a3e2dc502b None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:30.854953 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac13f57-fdd2-4f40-89dc-bc25fb3c8ec1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:30.855532 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac13f57-fdd2-4f40-89dc-bc25fb3c8ec1 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:30.855852 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac13f57-fdd2-4f40-89dc-bc25fb3c8ec1 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:30.867800 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac13f57-fdd2-4f40-89dc-bc25fb3c8ec1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:30.867800 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac13f57-fdd2-4f40-89dc-bc25fb3c8ec1 None admin] RBAC: Adding query filter params (role.id=0665869b71fa40b0b552efdd57faf023, scope.project.id=87b3e9d4b6474bd4a04ba59ef52bf3e2, user.id=d259178570864ffeb936ee10dfec6b28) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:59:30.867800 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac13f57-fdd2-4f40-89dc-bc25fb3c8ec1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:30.867800 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac13f57-fdd2-4f40-89dc-bc25fb3c8ec1 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:30.867800 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 329/686] 10.222.0.55 () {56 vars in 1374 bytes} [Fri Sep 23 14:59:30 2022] GET /identity/v3/role_assignments?user.id=d259178570864ffeb936ee10dfec6b28&scope.project.id=87b3e9d4b6474bd4a04ba59ef52bf3e2&role.id=0665869b71fa40b0b552efdd57faf023 => generated 608 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:59:30.869060 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac13f57-fdd2-4f40-89dc-bc25fb3c8ec1 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:30.973844 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07985c1d-a7e3-496e-a74b-1675392d5129 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:30.994451 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac13f57-fdd2-4f40-89dc-bc25fb3c8ec1 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:31.028415 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be14a5c-f869-4e64-b2e4-2b9a97d247ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:31.029296 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be14a5c-f869-4e64-b2e4-2b9a97d247ca None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:31.029524 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be14a5c-f869-4e64-b2e4-2b9a97d247ca None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:31.029935 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be14a5c-f869-4e64-b2e4-2b9a97d247ca None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:31.047420 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be14a5c-f869-4e64-b2e4-2b9a97d247ca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 14:59:31.052787 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-5be14a5c-f869-4e64-b2e4-2b9a97d247ca None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 23 14:59:31.052787 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 14:59:31.052787 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 14:59:31.052787 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 14:59:31.052787 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 14:59:31.052787 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 14:59:31.052787 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 14:59:31.052787 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 14:59:31.052787 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 14:59:31.052787 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 14:59:31.052787 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 14:59:31.052787 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 14:59:31.052787 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 14:59:31.052787 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 14:59:31.052787 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 14:59:31.052787 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 14:59:31.053792 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 14:59:31.053792 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 14:59:31.053792 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 14:59:31.053792 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 14:59:31.053792 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 14:59:31.053792 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 14:59:31.053792 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 14:59:31.053792 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 14:59:31.053792 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 14:59:31.053792 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 14:59:31.053792 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 14:59:31.053792 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Sep 23 14:59:31.053792 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 14:59:31.053792 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Sep 23 14:59:31.053792 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 14:59:31.053792 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 14:59:31.054639 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 14:59:31.054639 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Sep 23 14:59:31.054639 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 14:59:31.054639 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Sep 23 14:59:31.054639 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 14:59:31.054639 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 14:59:31.054639 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 14:59:31.054639 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 14:59:31.054639 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 14:59:31.054639 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 14:59:31.054639 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 23 14:59:31.054639 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Sep 23 14:59:31.054639 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 358/687] 10.222.0.55 () {56 vars in 1096 bytes} [Fri Sep 23 14:59:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 14:59:31.063916 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be14a5c-f869-4e64-b2e4-2b9a97d247ca None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:31.070373 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e215bad-e44d-4a3f-a011-b90d24bf05b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:31.071016 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e215bad-e44d-4a3f-a011-b90d24bf05b6 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:31.071399 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e215bad-e44d-4a3f-a011-b90d24bf05b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:31.071857 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e215bad-e44d-4a3f-a011-b90d24bf05b6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:31.110494 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:31.142294 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:31.174253 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e215bad-e44d-4a3f-a011-b90d24bf05b6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:31.175713 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e215bad-e44d-4a3f-a011-b90d24bf05b6 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:31.205175 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be14a5c-f869-4e64-b2e4-2b9a97d247ca None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:31.250935 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20499e20-f940-48ee-90cc-4f2672552874 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:31.251547 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20499e20-f940-48ee-90cc-4f2672552874 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:31.251826 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20499e20-f940-48ee-90cc-4f2672552874 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:31.252130 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20499e20-f940-48ee-90cc-4f2672552874 None admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:31.252903 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20499e20-f940-48ee-90cc-4f2672552874 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:59:31.253212 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20499e20-f940-48ee-90cc-4f2672552874 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:31.254683 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20499e20-f940-48ee-90cc-4f2672552874 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:31.264789 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 359/688] 10.222.0.55 () {56 vars in 1099 bytes} [Fri Sep 23 14:59:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 14:59:31.273306 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20499e20-f940-48ee-90cc-4f2672552874 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:31.298813 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 330/689] 10.222.0.55 () {58 vars in 1282 bytes} [Fri Sep 23 14:59:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 23 14:59:31.311633 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e215bad-e44d-4a3f-a011-b90d24bf05b6 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:31.417577 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:31.459201 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20499e20-f940-48ee-90cc-4f2672552874 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:31.489046 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:31.496458 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e215bad-e44d-4a3f-a011-b90d24bf05b6 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:31.524024 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2759a654-3481-411b-b8dd-5e3064c0272f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:31.525919 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2759a654-3481-411b-b8dd-5e3064c0272f None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:31.526436 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2759a654-3481-411b-b8dd-5e3064c0272f None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:31.527886 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2759a654-3481-411b-b8dd-5e3064c0272f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:31.547373 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2759a654-3481-411b-b8dd-5e3064c0272f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:31.549696 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2759a654-3481-411b-b8dd-5e3064c0272f None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:31.569744 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3013cb-bd32-44bc-af0f-a4eebd68ada4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:31.570485 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3013cb-bd32-44bc-af0f-a4eebd68ada4 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:31.570841 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3013cb-bd32-44bc-af0f-a4eebd68ada4 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:31.571183 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3013cb-bd32-44bc-af0f-a4eebd68ada4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:31.578394 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-2759a654-3481-411b-b8dd-5e3064c0272f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 14:59:31.578394 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 14:59:31.578394 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 14:59:31.578394 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 14:59:31.578394 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 14:59:31.578394 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 14:59:31.578394 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 14:59:31.578394 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 14:59:31.578394 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 14:59:31.578394 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 14:59:31.578394 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 14:59:31.578394 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 14:59:31.578394 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 14:59:31.578394 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 14:59:31.578394 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 14:59:31.578394 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 14:59:31.579570 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 14:59:31.579570 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 14:59:31.579570 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 14:59:31.579570 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 14:59:31.579570 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 14:59:31.579570 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 14:59:31.579570 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 14:59:31.579570 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Sep 23 14:59:31.579570 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 14:59:31.579570 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Sep 23 14:59:31.579570 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 14:59:31.579570 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 14:59:31.579570 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 14:59:31.579570 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Sep 23 14:59:31.579570 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 14:59:31.579570 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Sep 23 14:59:31.579570 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 14:59:31.580694 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 14:59:31.580694 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 14:59:31.580694 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Sep 23 14:59:31.580694 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 360/690] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Sep 23 14:59:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 14:59:31.590654 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2759a654-3481-411b-b8dd-5e3064c0272f None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:31.593857 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3013cb-bd32-44bc-af0f-a4eebd68ada4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:31.595616 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3013cb-bd32-44bc-af0f-a4eebd68ada4 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:31.612221 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application [None req-fc3013cb-bd32-44bc-af0f-a4eebd68ada4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 14:59:31.612221 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 14:59:31.612221 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 14:59:31.612221 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 14:59:31.612221 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 14:59:31.612221 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 14:59:31.612221 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 14:59:31.612221 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 14:59:31.612221 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 14:59:31.612221 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 14:59:31.612221 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 14:59:31.612221 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 14:59:31.612221 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 14:59:31.612221 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 14:59:31.612221 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 14:59:31.612221 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 14:59:31.613449 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 14:59:31.613449 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 14:59:31.613449 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 14:59:31.613449 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 14:59:31.613449 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 14:59:31.613449 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 14:59:31.613449 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 14:59:31.613449 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application with Lock( Sep 23 14:59:31.613449 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 14:59:31.613449 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._enter() Sep 23 14:59:31.613449 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 14:59:31.613449 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 14:59:31.613449 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 14:59:31.613449 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.creator() Sep 23 14:59:31.613449 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 14:59:31.613449 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application created_value = creator( Sep 23 14:59:31.613449 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 14:59:31.614800 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 14:59:31.614800 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 14:59:31.614800 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Sep 23 14:59:31.614800 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 331/691] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Sep 23 14:59:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 23 14:59:31.623751 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3013cb-bd32-44bc-af0f-a4eebd68ada4 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:31.741875 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3013cb-bd32-44bc-af0f-a4eebd68ada4 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:31.764439 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:31.780134 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d44354a-9602-48ab-8101-c451981e7f7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:31.780884 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d44354a-9602-48ab-8101-c451981e7f7e None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:31.781202 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d44354a-9602-48ab-8101-c451981e7f7e None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:31.781527 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d44354a-9602-48ab-8101-c451981e7f7e None admin] PATH_INFO: `/v3/domains` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:31.788002 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d44354a-9602-48ab-8101-c451981e7f7e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:59:31.788002 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d44354a-9602-48ab-8101-c451981e7f7e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:31.788002 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d44354a-9602-48ab-8101-c451981e7f7e None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:31.789892 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2759a654-3481-411b-b8dd-5e3064c0272f None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:31.798184 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 332/692] 10.222.0.55 () {56 vars in 1109 bytes} [Fri Sep 23 14:59:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:59:31.803740 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d44354a-9602-48ab-8101-c451981e7f7e None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:31.847101 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29bfe5b-c708-4a88-a60d-9c2f5f238669 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:31.848085 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29bfe5b-c708-4a88-a60d-9c2f5f238669 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:31.848528 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29bfe5b-c708-4a88-a60d-9c2f5f238669 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:31.848948 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29bfe5b-c708-4a88-a60d-9c2f5f238669 None admin] PATH_INFO: `/v3/domains` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:31.849858 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29bfe5b-c708-4a88-a60d-9c2f5f238669 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:59:31.850319 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29bfe5b-c708-4a88-a60d-9c2f5f238669 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:31.851939 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29bfe5b-c708-4a88-a60d-9c2f5f238669 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:31.867133 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 361/693] 10.222.0.55 () {56 vars in 1109 bytes} [Fri Sep 23 14:59:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:59:31.874882 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29bfe5b-c708-4a88-a60d-9c2f5f238669 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:31.917975 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:31.965590 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d44354a-9602-48ab-8101-c451981e7f7e None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:32.041207 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1043805-4d4e-4c0e-b090-2577f63eea27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:32.041896 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1043805-4d4e-4c0e-b090-2577f63eea27 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:32.042169 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1043805-4d4e-4c0e-b090-2577f63eea27 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:32.042450 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1043805-4d4e-4c0e-b090-2577f63eea27 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:32.049827 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29bfe5b-c708-4a88-a60d-9c2f5f238669 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:32.051437 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1043805-4d4e-4c0e-b090-2577f63eea27 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:32.053192 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1043805-4d4e-4c0e-b090-2577f63eea27 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:32.069529 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application [None req-f1043805-4d4e-4c0e-b090-2577f63eea27 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 14:59:32.069529 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 14:59:32.069529 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 14:59:32.069529 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 14:59:32.069529 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 14:59:32.069529 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 14:59:32.069529 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 14:59:32.069529 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 14:59:32.069529 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 14:59:32.069529 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 14:59:32.069529 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 14:59:32.069529 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 14:59:32.069529 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 23 14:59:32.069529 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 23 14:59:32.069529 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 23 14:59:32.070453 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 23 14:59:32.070453 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 14:59:32.070453 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 14:59:32.070453 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 14:59:32.070453 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 14:59:32.070453 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 14:59:32.070453 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 14:59:32.070453 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 14:59:32.070453 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application with Lock( Sep 23 14:59:32.070453 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 14:59:32.070453 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._enter() Sep 23 14:59:32.070453 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 14:59:32.070453 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 14:59:32.070453 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 14:59:32.070453 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.creator() Sep 23 14:59:32.070453 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 14:59:32.070453 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application created_value = creator( Sep 23 14:59:32.072664 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 23 14:59:32.072664 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 23 14:59:32.072664 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 23 14:59:32.072664 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 23 14:59:32.072664 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 23 14:59:32.072664 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 23 14:59:32.072664 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 14:59:32.072664 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Sep 23 14:59:32.076051 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 333/694] 10.222.0.55 () {56 vars in 1111 bytes} [Fri Sep 23 14:59:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 23 14:59:32.078376 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33381ddc-1729-4d7b-885d-6a5d09a5b0da None None] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:32.078661 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33381ddc-1729-4d7b-885d-6a5d09a5b0da None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:32.078934 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33381ddc-1729-4d7b-885d-6a5d09a5b0da None None] PATH_INFO: `/` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:32.084609 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 334/695] 10.222.0.55 () {54 vars in 927 bytes} [Fri Sep 23 14:59:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 14:59:32.085511 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33381ddc-1729-4d7b-885d-6a5d09a5b0da None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:32.101812 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d589a5-0d2e-40a5-b960-8ad09e676bf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:32.103529 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d589a5-0d2e-40a5-b960-8ad09e676bf1 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:32.104059 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d589a5-0d2e-40a5-b960-8ad09e676bf1 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:32.104572 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d589a5-0d2e-40a5-b960-8ad09e676bf1 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:32.112926 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d589a5-0d2e-40a5-b960-8ad09e676bf1 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:32.114670 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d589a5-0d2e-40a5-b960-8ad09e676bf1 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:32.115248 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:32.129950 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-61d589a5-0d2e-40a5-b960-8ad09e676bf1 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 23 14:59:32.129950 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 14:59:32.129950 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 14:59:32.129950 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 14:59:32.129950 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 14:59:32.129950 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 14:59:32.129950 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 14:59:32.129950 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 14:59:32.129950 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 14:59:32.129950 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 14:59:32.129950 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 14:59:32.129950 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 14:59:32.129950 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 23 14:59:32.129950 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 23 14:59:32.129950 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 23 14:59:32.129950 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 23 14:59:32.131458 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 14:59:32.131458 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 14:59:32.131458 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 23 14:59:32.131458 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 14:59:32.131458 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 23 14:59:32.131458 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 14:59:32.131458 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 14:59:32.131458 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 14:59:32.131458 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 14:59:32.131458 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 14:59:32.131458 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 14:59:32.131458 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Sep 23 14:59:32.131458 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 14:59:32.131458 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Sep 23 14:59:32.131458 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 14:59:32.131458 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 14:59:32.131458 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 14:59:32.131458 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Sep 23 14:59:32.132714 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 14:59:32.132714 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Sep 23 14:59:32.132714 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 23 14:59:32.132714 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 23 14:59:32.132714 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 23 14:59:32.132714 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 23 14:59:32.132714 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 23 14:59:32.132714 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 23 14:59:32.132714 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 23 14:59:32.132714 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Sep 23 14:59:32.135320 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 362/696] 10.222.0.55 () {56 vars in 1093 bytes} [Fri Sep 23 14:59:31 2022] GET /identity/v3/users/nova => generated 82 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 23 14:59:32.137051 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5c484d-22f6-444f-a6a7-e8473730ee59 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:32.137546 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5c484d-22f6-444f-a6a7-e8473730ee59 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:32.138040 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5c484d-22f6-444f-a6a7-e8473730ee59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:32.159390 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-0c5c484d-22f6-444f-a6a7-e8473730ee59 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 14:59:32.161132 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-0c5c484d-22f6-444f-a6a7-e8473730ee59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 14:59:32.204160 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33381ddc-1729-4d7b-885d-6a5d09a5b0da None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:32.242158 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69112699-fc40-4ca5-b86b-613a71143089 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:32.242764 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69112699-fc40-4ca5-b86b-613a71143089 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:32.243003 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69112699-fc40-4ca5-b86b-613a71143089 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:32.243903 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69112699-fc40-4ca5-b86b-613a71143089 None admin] PATH_INFO: `/v3/projects` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:32.244660 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69112699-fc40-4ca5-b86b-613a71143089 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:59:32.244954 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69112699-fc40-4ca5-b86b-613a71143089 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:32.246490 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69112699-fc40-4ca5-b86b-613a71143089 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:32.256741 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 363/697] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Sep 23 14:59:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 14:59:32.257944 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5c484d-22f6-444f-a6a7-e8473730ee59 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:32.271179 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 335/698] 10.222.0.55 () {56 vars in 1148 bytes} [Fri Sep 23 14:59:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:59:32.284457 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69112699-fc40-4ca5-b86b-613a71143089 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:32.287644 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:32.412421 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5c484d-22f6-444f-a6a7-e8473730ee59 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:32.459337 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bb5b92-bc18-46df-bd1d-d8bad639016b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:32.459882 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:32.460018 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bb5b92-bc18-46df-bd1d-d8bad639016b None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:32.460256 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bb5b92-bc18-46df-bd1d-d8bad639016b None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:32.460534 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bb5b92-bc18-46df-bd1d-d8bad639016b None admin] PATH_INFO: `/v3/users` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:32.461270 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bb5b92-bc18-46df-bd1d-d8bad639016b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:59:32.461574 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bb5b92-bc18-46df-bd1d-d8bad639016b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:32.463573 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bb5b92-bc18-46df-bd1d-d8bad639016b None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:32.485871 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69112699-fc40-4ca5-b86b-613a71143089 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:32.486613 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 364/699] 10.222.0.55 () {56 vars in 1133 bytes} [Fri Sep 23 14:59:32 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:59:32.493477 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bb5b92-bc18-46df-bd1d-d8bad639016b None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:32.567971 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725e5215-26b1-4c5a-83b0-5be4761fbdfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:32.568866 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725e5215-26b1-4c5a-83b0-5be4761fbdfa None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:32.569319 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725e5215-26b1-4c5a-83b0-5be4761fbdfa None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:32.569813 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725e5215-26b1-4c5a-83b0-5be4761fbdfa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:32.570681 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725e5215-26b1-4c5a-83b0-5be4761fbdfa None admin] RBAC: Adding query filter params (role.id=0665869b71fa40b0b552efdd57faf023, scope.project.id=87b3e9d4b6474bd4a04ba59ef52bf3e2, user.id=2228c0b49a0249bbaf524a8c4e9e108b) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:59:32.571130 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725e5215-26b1-4c5a-83b0-5be4761fbdfa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:32.574978 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725e5215-26b1-4c5a-83b0-5be4761fbdfa None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:32.586646 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 336/700] 10.222.0.55 () {56 vars in 1374 bytes} [Fri Sep 23 14:59:32 2022] GET /identity/v3/role_assignments?user.id=2228c0b49a0249bbaf524a8c4e9e108b&scope.project.id=87b3e9d4b6474bd4a04ba59ef52bf3e2&role.id=0665869b71fa40b0b552efdd57faf023 => generated 608 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 14:59:32.627031 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:32.642811 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725e5215-26b1-4c5a-83b0-5be4761fbdfa None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:32.655253 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bb5b92-bc18-46df-bd1d-d8bad639016b None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:32.698494 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198aceea-309d-44cd-a7d2-32a937a41d13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:32.699591 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198aceea-309d-44cd-a7d2-32a937a41d13 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:32.700837 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198aceea-309d-44cd-a7d2-32a937a41d13 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:32.701439 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198aceea-309d-44cd-a7d2-32a937a41d13 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:32.830327 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198aceea-309d-44cd-a7d2-32a937a41d13 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:32.832297 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198aceea-309d-44cd-a7d2-32a937a41d13 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:32.863069 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725e5215-26b1-4c5a-83b0-5be4761fbdfa None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:32.906806 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99514ac5-b841-42a2-99a1-d19d880b1d38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:32.908039 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99514ac5-b841-42a2-99a1-d19d880b1d38 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:32.908151 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99514ac5-b841-42a2-99a1-d19d880b1d38 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:32.908432 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99514ac5-b841-42a2-99a1-d19d880b1d38 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:32.925499 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99514ac5-b841-42a2-99a1-d19d880b1d38 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:32.926965 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99514ac5-b841-42a2-99a1-d19d880b1d38 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:32.933445 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:32.949986 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application [None req-99514ac5-b841-42a2-99a1-d19d880b1d38 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 14:59:32.949986 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 14:59:32.949986 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 14:59:32.949986 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 14:59:32.949986 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 14:59:32.949986 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 14:59:32.949986 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 14:59:32.949986 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 14:59:32.949986 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 14:59:32.949986 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 14:59:32.949986 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 14:59:32.949986 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 14:59:32.949986 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 14:59:32.949986 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 14:59:32.949986 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 14:59:32.949986 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 14:59:32.950994 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 14:59:32.950994 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 14:59:32.950994 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 14:59:32.950994 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 14:59:32.950994 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 14:59:32.950994 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 14:59:32.950994 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 14:59:32.950994 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application with Lock( Sep 23 14:59:32.950994 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 14:59:32.950994 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._enter() Sep 23 14:59:32.950994 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 14:59:32.950994 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 14:59:32.950994 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 14:59:32.950994 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.creator() Sep 23 14:59:32.950994 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 14:59:32.950994 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application created_value = creator( Sep 23 14:59:32.950994 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 14:59:32.951898 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 14:59:32.951898 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 14:59:32.951898 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Sep 23 14:59:32.951898 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 337/701] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Sep 23 14:59:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 23 14:59:32.956956 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99514ac5-b841-42a2-99a1-d19d880b1d38 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:32.961242 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:32.965342 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 365/702] 10.222.0.55 () {58 vars in 1282 bytes} [Fri Sep 23 14:59:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 472 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 23 14:59:32.970975 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198aceea-309d-44cd-a7d2-32a937a41d13 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:33.095145 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198aceea-309d-44cd-a7d2-32a937a41d13 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:33.110404 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99514ac5-b841-42a2-99a1-d19d880b1d38 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:33.144237 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244d5ea8-ec83-477e-8edd-fbd17f54efa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:33.145473 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244d5ea8-ec83-477e-8edd-fbd17f54efa1 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:33.145858 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244d5ea8-ec83-477e-8edd-fbd17f54efa1 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:33.146446 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244d5ea8-ec83-477e-8edd-fbd17f54efa1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:33.158661 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2e3473-ed02-4447-8540-54e5b6564487 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:33.159512 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2e3473-ed02-4447-8540-54e5b6564487 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:33.159933 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2e3473-ed02-4447-8540-54e5b6564487 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:33.160376 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2e3473-ed02-4447-8540-54e5b6564487 None admin] PATH_INFO: `/v3/domains` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:33.161069 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244d5ea8-ec83-477e-8edd-fbd17f54efa1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:33.161342 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2e3473-ed02-4447-8540-54e5b6564487 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:59:33.161780 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2e3473-ed02-4447-8540-54e5b6564487 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:33.162917 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244d5ea8-ec83-477e-8edd-fbd17f54efa1 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:33.163440 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2e3473-ed02-4447-8540-54e5b6564487 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:33.176713 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 338/703] 10.222.0.55 () {56 vars in 1109 bytes} [Fri Sep 23 14:59:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 14:59:33.180565 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-244d5ea8-ec83-477e-8edd-fbd17f54efa1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 14:59:33.180565 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 14:59:33.180565 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 14:59:33.180565 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 14:59:33.180565 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 14:59:33.180565 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 14:59:33.180565 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 14:59:33.180565 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 14:59:33.180565 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 14:59:33.180565 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 14:59:33.180565 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 14:59:33.180565 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 14:59:33.180565 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 14:59:33.180565 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 14:59:33.180565 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 14:59:33.180565 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 14:59:33.181574 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 14:59:33.181574 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 14:59:33.181574 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 14:59:33.181574 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 14:59:33.181574 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 14:59:33.181574 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 14:59:33.181574 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 14:59:33.181574 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Sep 23 14:59:33.181574 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 14:59:33.181574 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Sep 23 14:59:33.181574 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 14:59:33.181574 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 14:59:33.181574 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 14:59:33.181574 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Sep 23 14:59:33.181574 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 14:59:33.181574 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Sep 23 14:59:33.181574 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 14:59:33.182487 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 14:59:33.182487 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 14:59:33.182487 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Sep 23 14:59:33.182666 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2e3473-ed02-4447-8540-54e5b6564487 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:33.185111 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 366/704] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Sep 23 14:59:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 23 14:59:33.189906 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244d5ea8-ec83-477e-8edd-fbd17f54efa1 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:33.237294 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:33.258636 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:33.322740 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2e3473-ed02-4447-8540-54e5b6564487 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:33.331184 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244d5ea8-ec83-477e-8edd-fbd17f54efa1 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:33.375134 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7129c11-e790-41f7-acfe-3bd714c664b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:33.376156 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7129c11-e790-41f7-acfe-3bd714c664b1 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:33.376642 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7129c11-e790-41f7-acfe-3bd714c664b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:33.377204 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7129c11-e790-41f7-acfe-3bd714c664b1 None admin] PATH_INFO: `/v3/domains` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:33.378064 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7129c11-e790-41f7-acfe-3bd714c664b1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:59:33.378567 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7129c11-e790-41f7-acfe-3bd714c664b1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:33.380440 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7129c11-e790-41f7-acfe-3bd714c664b1 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:33.395307 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 367/705] 10.222.0.55 () {56 vars in 1109 bytes} [Fri Sep 23 14:59:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 14:59:33.402987 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7129c11-e790-41f7-acfe-3bd714c664b1 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:33.428708 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfd61f9-bb0c-42e2-b685-e6fc0e1f5021 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:33.429336 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfd61f9-bb0c-42e2-b685-e6fc0e1f5021 None admin] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:33.429614 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfd61f9-bb0c-42e2-b685-e6fc0e1f5021 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:33.429880 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfd61f9-bb0c-42e2-b685-e6fc0e1f5021 None admin] PATH_INFO: `/v3/users` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:33.430540 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfd61f9-bb0c-42e2-b685-e6fc0e1f5021 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:33.432039 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfd61f9-bb0c-42e2-b685-e6fc0e1f5021 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:33.507810 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:33.518354 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7129c11-e790-41f7-acfe-3bd714c664b1 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:33.554972 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0631052-13f8-4834-841d-2418aca1f750 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:33.556167 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0631052-13f8-4834-841d-2418aca1f750 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:33.556451 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0631052-13f8-4834-841d-2418aca1f750 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:33.556754 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0631052-13f8-4834-841d-2418aca1f750 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:33.564456 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0631052-13f8-4834-841d-2418aca1f750 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:33.565913 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0631052-13f8-4834-841d-2418aca1f750 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:33.578701 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-c0631052-13f8-4834-841d-2418aca1f750 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 14:59:33.578701 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 14:59:33.578701 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 14:59:33.578701 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 14:59:33.578701 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 14:59:33.578701 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 14:59:33.578701 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 14:59:33.578701 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 14:59:33.578701 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 14:59:33.578701 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 14:59:33.578701 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 14:59:33.578701 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 14:59:33.578701 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 23 14:59:33.578701 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 23 14:59:33.578701 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 23 14:59:33.579859 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 23 14:59:33.579859 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 14:59:33.579859 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 14:59:33.579859 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 14:59:33.579859 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 14:59:33.579859 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 14:59:33.579859 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 14:59:33.579859 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 14:59:33.579859 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Sep 23 14:59:33.579859 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 14:59:33.579859 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Sep 23 14:59:33.579859 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 14:59:33.579859 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 14:59:33.579859 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 14:59:33.579859 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Sep 23 14:59:33.579859 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 14:59:33.579859 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Sep 23 14:59:33.580968 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 23 14:59:33.580968 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 23 14:59:33.580968 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 23 14:59:33.580968 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 23 14:59:33.580968 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 23 14:59:33.580968 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 23 14:59:33.580968 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 14:59:33.580968 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Sep 23 14:59:33.580968 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 368/706] 10.222.0.55 () {56 vars in 1111 bytes} [Fri Sep 23 14:59:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 14:59:33.581589 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9fdc50-8f46-4080-ad62-f111ae450330 None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:33.581589 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9fdc50-8f46-4080-ad62-f111ae450330 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:33.581743 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9fdc50-8f46-4080-ad62-f111ae450330 None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:33.582902 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 369/707] 10.222.0.55 () {54 vars in 927 bytes} [Fri Sep 23 14:59:33 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 14:59:33.588502 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9fdc50-8f46-4080-ad62-f111ae450330 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:33.604087 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 339/708] 10.222.0.55 () {60 vars in 1132 bytes} [Fri Sep 23 14:59:33 2022] POST /identity/v3/users => generated 254 bytes in 418 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 14:59:33.605900 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1c1625-bbb3-4e20-b0af-442a24609ab4 None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:33.606001 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1c1625-bbb3-4e20-b0af-442a24609ab4 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:33.606282 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1c1625-bbb3-4e20-b0af-442a24609ab4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:33.661729 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-ea1c1625-bbb3-4e20-b0af-442a24609ab4 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 14:59:33.663010 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-ea1c1625-bbb3-4e20-b0af-442a24609ab4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 14:59:33.735540 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9fdc50-8f46-4080-ad62-f111ae450330 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:33.768152 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 340/709] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Sep 23 14:59:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 14:59:33.768152 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82335527-30dc-4db8-9e9e-5565fb4ce10a None None] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:33.768152 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82335527-30dc-4db8-9e9e-5565fb4ce10a None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:33.768152 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82335527-30dc-4db8-9e9e-5565fb4ce10a None None] PATH_INFO: `/` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:33.772058 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 341/710] 10.222.0.55 () {54 vars in 927 bytes} [Fri Sep 23 14:59:33 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 14:59:33.773953 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38a7b87-5adf-4cc9-b453-358c30cf6be0 None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:33.774348 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38a7b87-5adf-4cc9-b453-358c30cf6be0 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:33.775245 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38a7b87-5adf-4cc9-b453-358c30cf6be0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:33.803119 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-d38a7b87-5adf-4cc9-b453-358c30cf6be0 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 14:59:33.804062 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268aaac5-be29-43af-833e-afde01e6c932 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:33.804768 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-d38a7b87-5adf-4cc9-b453-358c30cf6be0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 14:59:33.805188 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268aaac5-be29-43af-833e-afde01e6c932 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:33.805686 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268aaac5-be29-43af-833e-afde01e6c932 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:33.806099 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268aaac5-be29-43af-833e-afde01e6c932 None admin] PATH_INFO: `/v3/projects` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:33.806953 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268aaac5-be29-43af-833e-afde01e6c932 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:59:33.807650 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268aaac5-be29-43af-833e-afde01e6c932 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:33.810215 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268aaac5-be29-43af-833e-afde01e6c932 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:33.838762 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 370/711] 10.222.0.55 () {56 vars in 1148 bytes} [Fri Sep 23 14:59:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 14:59:33.845495 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268aaac5-be29-43af-833e-afde01e6c932 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:33.864951 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:33.869143 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:33.894877 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 342/712] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Sep 23 14:59:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 23 14:59:33.974030 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268aaac5-be29-43af-833e-afde01e6c932 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:34.014420 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af82d180-629d-4432-93e4-462fdd8ceb1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:34.015209 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af82d180-629d-4432-93e4-462fdd8ceb1a None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:34.015644 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af82d180-629d-4432-93e4-462fdd8ceb1a None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:34.016042 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af82d180-629d-4432-93e4-462fdd8ceb1a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:34.016963 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af82d180-629d-4432-93e4-462fdd8ceb1a None admin] RBAC: Adding query filter params (role.id=8e2919e79df040e8926f21607818b58c, scope.project.id=87b3e9d4b6474bd4a04ba59ef52bf3e2, user.id=b3c6c7a1325a47be924268e1aa1a7a18) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:59:34.017379 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af82d180-629d-4432-93e4-462fdd8ceb1a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:34.019002 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af82d180-629d-4432-93e4-462fdd8ceb1a None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:34.027205 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 371/713] 10.222.0.55 () {56 vars in 1374 bytes} [Fri Sep 23 14:59:33 2022] GET /identity/v3/role_assignments?user.id=b3c6c7a1325a47be924268e1aa1a7a18&scope.project.id=87b3e9d4b6474bd4a04ba59ef52bf3e2&role.id=8e2919e79df040e8926f21607818b58c => generated 259 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 14:59:34.110648 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38a7b87-5adf-4cc9-b453-358c30cf6be0 None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:34.213763 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af82d180-629d-4432-93e4-462fdd8ceb1a None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:34.249520 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38a7b87-5adf-4cc9-b453-358c30cf6be0 None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:34.289884 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef4a323-942a-452d-b84c-9d05b43ae8e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:34.290738 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef4a323-942a-452d-b84c-9d05b43ae8e5 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:34.290860 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef4a323-942a-452d-b84c-9d05b43ae8e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:34.291039 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef4a323-942a-452d-b84c-9d05b43ae8e5 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:34.291619 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef4a323-942a-452d-b84c-9d05b43ae8e5 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:34.293099 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef4a323-942a-452d-b84c-9d05b43ae8e5 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:34.304721 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application [None req-8ef4a323-942a-452d-b84c-9d05b43ae8e5 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 23 14:59:34.304721 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 14:59:34.304721 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 23 14:59:34.304721 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 23 14:59:34.304721 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 23 14:59:34.304721 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 23 14:59:34.304721 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 23 14:59:34.304721 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 23 14:59:34.304721 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 23 14:59:34.304721 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Sep 23 14:59:34.304721 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 23 14:59:34.304721 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Sep 23 14:59:34.304721 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 14:59:34.304721 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 14:59:34.304721 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 14:59:34.304721 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 14:59:34.304721 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 14:59:34.306479 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 14:59:34.306479 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 14:59:34.306479 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 14:59:34.306479 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 14:59:34.306479 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 14:59:34.306479 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 14:59:34.306479 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 23 14:59:34.306479 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 23 14:59:34.306479 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 23 14:59:34.306479 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 23 14:59:34.306479 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 14:59:34.306479 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 14:59:34.306479 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 14:59:34.306479 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 14:59:34.306479 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 14:59:34.306479 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 14:59:34.307421 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 14:59:34.307421 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application with Lock( Sep 23 14:59:34.307421 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 14:59:34.307421 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._enter() Sep 23 14:59:34.307421 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 14:59:34.307421 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 14:59:34.307421 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 14:59:34.307421 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.creator() Sep 23 14:59:34.307421 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 14:59:34.307421 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application created_value = creator( Sep 23 14:59:34.307421 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 23 14:59:34.307421 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 23 14:59:34.307421 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 23 14:59:34.307421 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Sep 23 14:59:34.311330 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 343/714] 10.222.0.55 () {56 vars in 1129 bytes} [Fri Sep 23 14:59:34 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 14:59:34.312635 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef4a323-942a-452d-b84c-9d05b43ae8e5 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:34.318421 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:34.370977 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af82d180-629d-4432-93e4-462fdd8ceb1a None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:34.392759 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:34.441441 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f32de6-af91-4122-b599-ad053f5aa9a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:34.442083 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f32de6-af91-4122-b599-ad053f5aa9a7 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:34.442358 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f32de6-af91-4122-b599-ad053f5aa9a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:34.442627 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f32de6-af91-4122-b599-ad053f5aa9a7 None admin] PATH_INFO: `/v3/services/network` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:34.443206 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f32de6-af91-4122-b599-ad053f5aa9a7 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:34.444582 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f32de6-af91-4122-b599-ad053f5aa9a7 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:34.466131 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-e4f32de6-af91-4122-b599-ad053f5aa9a7 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 23 14:59:34.466131 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 14:59:34.466131 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 23 14:59:34.466131 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 23 14:59:34.466131 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 23 14:59:34.466131 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 23 14:59:34.466131 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 23 14:59:34.466131 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 23 14:59:34.466131 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 23 14:59:34.466131 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Sep 23 14:59:34.466131 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 23 14:59:34.466131 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Sep 23 14:59:34.466131 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 14:59:34.466131 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 14:59:34.466131 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 14:59:34.466131 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 14:59:34.466131 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 14:59:34.467397 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 14:59:34.467397 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 14:59:34.467397 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 14:59:34.467397 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 14:59:34.467397 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 14:59:34.467397 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 14:59:34.467397 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 23 14:59:34.467397 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 23 14:59:34.467397 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 23 14:59:34.467397 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 23 14:59:34.467397 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 14:59:34.467397 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 14:59:34.467397 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 14:59:34.467397 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 14:59:34.467397 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 14:59:34.467397 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 14:59:34.468387 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 14:59:34.468387 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Sep 23 14:59:34.468387 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 14:59:34.468387 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Sep 23 14:59:34.468387 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 14:59:34.468387 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 14:59:34.468387 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 14:59:34.468387 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Sep 23 14:59:34.468387 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 14:59:34.468387 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Sep 23 14:59:34.468387 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 23 14:59:34.468387 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 23 14:59:34.468387 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 23 14:59:34.468387 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Sep 23 14:59:34.474173 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 372/715] 10.222.0.55 () {56 vars in 1111 bytes} [Fri Sep 23 14:59:34 2022] GET /identity/v3/services/network => generated 88 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 14:59:34.481474 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f32de6-af91-4122-b599-ad053f5aa9a7 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:34.507641 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef4a323-942a-452d-b84c-9d05b43ae8e5 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:34.577688 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd67c78-ea5e-48c0-8ca1-d0dbf42b4ebd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:34.578344 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd67c78-ea5e-48c0-8ca1-d0dbf42b4ebd None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:34.578694 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd67c78-ea5e-48c0-8ca1-d0dbf42b4ebd None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:34.578962 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd67c78-ea5e-48c0-8ca1-d0dbf42b4ebd None admin] PATH_INFO: `/v3/services` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:34.579585 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd67c78-ea5e-48c0-8ca1-d0dbf42b4ebd None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:59:34.579882 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd67c78-ea5e-48c0-8ca1-d0dbf42b4ebd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:34.581280 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd67c78-ea5e-48c0-8ca1-d0dbf42b4ebd None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:34.596621 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 344/716] 10.222.0.55 () {56 vars in 1124 bytes} [Fri Sep 23 14:59:34 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:59:34.599844 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd67c78-ea5e-48c0-8ca1-d0dbf42b4ebd None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:34.657692 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f32de6-af91-4122-b599-ad053f5aa9a7 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:34.700098 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48eb108-ce42-4eab-85fb-a79050d5907e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:34.701083 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48eb108-ce42-4eab-85fb-a79050d5907e None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:34.701621 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48eb108-ce42-4eab-85fb-a79050d5907e None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:34.702078 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48eb108-ce42-4eab-85fb-a79050d5907e None admin] PATH_INFO: `/v3/services` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:34.704670 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48eb108-ce42-4eab-85fb-a79050d5907e None admin] RBAC: Adding query filter params (name=network) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:59:34.709948 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48eb108-ce42-4eab-85fb-a79050d5907e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:34.709948 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48eb108-ce42-4eab-85fb-a79050d5907e None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:34.714607 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 373/717] 10.222.0.55 () {56 vars in 1112 bytes} [Fri Sep 23 14:59:34 2022] GET /identity/v3/services?name=network => generated 124 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:59:34.722425 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48eb108-ce42-4eab-85fb-a79050d5907e None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:34.725599 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:34.836051 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd67c78-ea5e-48c0-8ca1-d0dbf42b4ebd None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:34.836599 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48eb108-ce42-4eab-85fb-a79050d5907e None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:34.850012 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:34.875410 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defa8183-226d-43e0-9749-1da5ae0d519f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:34.876060 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defa8183-226d-43e0-9749-1da5ae0d519f None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:34.876348 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defa8183-226d-43e0-9749-1da5ae0d519f None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:34.876623 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defa8183-226d-43e0-9749-1da5ae0d519f None admin] PATH_INFO: `/v3/services` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:34.877255 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defa8183-226d-43e0-9749-1da5ae0d519f None admin] RBAC: Adding query filter params (type=network) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:59:34.877562 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defa8183-226d-43e0-9749-1da5ae0d519f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:34.878968 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defa8183-226d-43e0-9749-1da5ae0d519f None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:34.886752 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 374/718] 10.222.0.55 () {56 vars in 1112 bytes} [Fri Sep 23 14:59:34 2022] GET /identity/v3/services?type=network => generated 124 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:59:34.922794 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854c6832-104b-46f5-b906-710f0e4620b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:34.923491 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854c6832-104b-46f5-b906-710f0e4620b9 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:34.923775 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854c6832-104b-46f5-b906-710f0e4620b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:34.924090 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854c6832-104b-46f5-b906-710f0e4620b9 None admin] PATH_INFO: `/v3/services` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:34.924755 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854c6832-104b-46f5-b906-710f0e4620b9 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:59:34.925091 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854c6832-104b-46f5-b906-710f0e4620b9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:34.926521 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854c6832-104b-46f5-b906-710f0e4620b9 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:34.937592 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 345/719] 10.222.0.55 () {56 vars in 1124 bytes} [Fri Sep 23 14:59:34 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 14:59:35.380399 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b80b21-6d67-4eb3-9789-36be250b70d2 None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:35.380956 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b80b21-6d67-4eb3-9789-36be250b70d2 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:35.380956 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b80b21-6d67-4eb3-9789-36be250b70d2 None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:35.382520 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 375/720] 10.222.0.55 () {54 vars in 927 bytes} [Fri Sep 23 14:59:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 14:59:35.388542 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc132b2-22d2-4a92-b165-052782595a46 None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:35.388957 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc132b2-22d2-4a92-b165-052782595a46 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:35.389180 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc132b2-22d2-4a92-b165-052782595a46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:35.409135 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-cdc132b2-22d2-4a92-b165-052782595a46 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 14:59:35.410331 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-cdc132b2-22d2-4a92-b165-052782595a46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 14:59:35.483812 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 346/721] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Sep 23 14:59:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 23 14:59:35.756238 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b80b21-6d67-4eb3-9789-36be250b70d2 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:35.845674 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:35.894008 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b80b21-6d67-4eb3-9789-36be250b70d2 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:35.933602 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3cb51a-e018-4d4f-b58b-c43fddd27177 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:35.934234 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3cb51a-e018-4d4f-b58b-c43fddd27177 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:35.934511 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3cb51a-e018-4d4f-b58b-c43fddd27177 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:35.934779 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3cb51a-e018-4d4f-b58b-c43fddd27177 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:35.945367 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3cb51a-e018-4d4f-b58b-c43fddd27177 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 14:59:35.952296 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-7a3cb51a-e018-4d4f-b58b-c43fddd27177 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 23 14:59:35.952296 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 14:59:35.952296 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 14:59:35.952296 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 14:59:35.952296 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 14:59:35.952296 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 14:59:35.952296 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 14:59:35.952296 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 14:59:35.952296 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 14:59:35.952296 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 14:59:35.952296 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 14:59:35.952296 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 14:59:35.952296 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 14:59:35.952296 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 14:59:35.952296 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 14:59:35.952296 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 14:59:35.955219 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 14:59:35.955219 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 14:59:35.955219 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 14:59:35.955219 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 14:59:35.955219 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 14:59:35.955219 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 14:59:35.955219 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 14:59:35.955219 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 14:59:35.955219 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 14:59:35.955219 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 14:59:35.955219 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 14:59:35.955219 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Sep 23 14:59:35.955219 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 14:59:35.955219 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Sep 23 14:59:35.955219 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 14:59:35.955219 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 14:59:35.958494 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 14:59:35.958494 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Sep 23 14:59:35.958494 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 14:59:35.958494 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Sep 23 14:59:35.958494 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 14:59:35.958494 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 14:59:35.958494 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 14:59:35.958494 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 14:59:35.958494 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 14:59:35.958494 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 14:59:35.958494 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 23 14:59:35.958494 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Sep 23 14:59:35.958494 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 376/722] 10.222.0.55 () {56 vars in 1096 bytes} [Fri Sep 23 14:59:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 14:59:35.959741 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc132b2-22d2-4a92-b165-052782595a46 None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:36.099298 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:36.102445 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc132b2-22d2-4a92-b165-052782595a46 None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:36.151842 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f59f1f9-1e97-48bc-8738-ef697a7daee6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:36.152490 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f59f1f9-1e97-48bc-8738-ef697a7daee6 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:36.152770 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f59f1f9-1e97-48bc-8738-ef697a7daee6 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:36.153043 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f59f1f9-1e97-48bc-8738-ef697a7daee6 None admin] PATH_INFO: `/v3/roles` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:36.153916 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f59f1f9-1e97-48bc-8738-ef697a7daee6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:59:36.154215 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f59f1f9-1e97-48bc-8738-ef697a7daee6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:36.155577 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f59f1f9-1e97-48bc-8738-ef697a7daee6 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:36.171643 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 347/723] 10.222.0.55 () {56 vars in 1099 bytes} [Fri Sep 23 14:59:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:59:36.177295 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3cb51a-e018-4d4f-b58b-c43fddd27177 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:36.307198 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3cb51a-e018-4d4f-b58b-c43fddd27177 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:36.337517 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:36.347213 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a5e86c-1c5d-4f58-a3e4-95404722967b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:36.358896 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a5e86c-1c5d-4f58-a3e4-95404722967b None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:36.358896 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a5e86c-1c5d-4f58-a3e4-95404722967b None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:36.358896 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a5e86c-1c5d-4f58-a3e4-95404722967b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:36.361025 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a5e86c-1c5d-4f58-a3e4-95404722967b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:36.362637 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a5e86c-1c5d-4f58-a3e4-95404722967b None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:36.378404 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-99a5e86c-1c5d-4f58-a3e4-95404722967b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 14:59:36.378404 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 14:59:36.378404 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 14:59:36.378404 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 14:59:36.378404 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 14:59:36.378404 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 14:59:36.378404 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 14:59:36.378404 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 14:59:36.378404 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 14:59:36.378404 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 14:59:36.378404 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 14:59:36.378404 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 14:59:36.378404 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 14:59:36.378404 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 14:59:36.378404 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 14:59:36.378404 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 14:59:36.379599 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 14:59:36.379599 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 14:59:36.379599 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 14:59:36.379599 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 14:59:36.379599 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 14:59:36.379599 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 14:59:36.379599 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 14:59:36.379599 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Sep 23 14:59:36.379599 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 14:59:36.379599 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Sep 23 14:59:36.379599 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 14:59:36.379599 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 14:59:36.379599 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 14:59:36.379599 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Sep 23 14:59:36.379599 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 14:59:36.379599 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Sep 23 14:59:36.379599 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 14:59:36.381284 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 14:59:36.381284 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 14:59:36.381284 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Sep 23 14:59:36.382989 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 377/724] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Sep 23 14:59:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 14:59:36.388449 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f59f1f9-1e97-48bc-8738-ef697a7daee6 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:36.521371 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f59f1f9-1e97-48bc-8738-ef697a7daee6 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:36.569855 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3285c0c3-27f1-4a54-ab37-bc420062ffe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:36.570468 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3285c0c3-27f1-4a54-ab37-bc420062ffe5 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:36.570743 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3285c0c3-27f1-4a54-ab37-bc420062ffe5 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:36.571056 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3285c0c3-27f1-4a54-ab37-bc420062ffe5 None admin] PATH_INFO: `/v3/domains` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:36.571902 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3285c0c3-27f1-4a54-ab37-bc420062ffe5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:59:36.572232 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3285c0c3-27f1-4a54-ab37-bc420062ffe5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:36.573690 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3285c0c3-27f1-4a54-ab37-bc420062ffe5 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:36.591203 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 348/725] 10.222.0.55 () {56 vars in 1109 bytes} [Fri Sep 23 14:59:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:59:36.596656 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a5e86c-1c5d-4f58-a3e4-95404722967b None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:36.746309 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a5e86c-1c5d-4f58-a3e4-95404722967b None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:36.801638 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0cdb2f-dfc1-4f1a-9431-e27cf1fd1ae2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:36.802297 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0cdb2f-dfc1-4f1a-9431-e27cf1fd1ae2 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:36.802587 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0cdb2f-dfc1-4f1a-9431-e27cf1fd1ae2 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:36.802859 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0cdb2f-dfc1-4f1a-9431-e27cf1fd1ae2 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:36.813215 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0cdb2f-dfc1-4f1a-9431-e27cf1fd1ae2 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:36.814722 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0cdb2f-dfc1-4f1a-9431-e27cf1fd1ae2 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:36.833532 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-4f0cdb2f-dfc1-4f1a-9431-e27cf1fd1ae2 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 23 14:59:36.833532 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 14:59:36.833532 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 14:59:36.833532 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 14:59:36.833532 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 14:59:36.833532 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 14:59:36.833532 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 14:59:36.833532 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 14:59:36.833532 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 14:59:36.833532 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 14:59:36.833532 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 14:59:36.833532 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 14:59:36.833532 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 23 14:59:36.833532 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 23 14:59:36.833532 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 23 14:59:36.833532 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 23 14:59:36.834561 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 14:59:36.834561 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 14:59:36.834561 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 23 14:59:36.834561 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 14:59:36.834561 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 23 14:59:36.834561 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 14:59:36.834561 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 14:59:36.834561 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 14:59:36.834561 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 14:59:36.834561 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 14:59:36.834561 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 14:59:36.834561 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Sep 23 14:59:36.834561 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 14:59:36.834561 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Sep 23 14:59:36.834561 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 14:59:36.834561 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 14:59:36.834561 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 14:59:36.834561 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Sep 23 14:59:36.835500 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 14:59:36.835500 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Sep 23 14:59:36.835500 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 23 14:59:36.835500 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 23 14:59:36.835500 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 23 14:59:36.835500 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 23 14:59:36.835500 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 23 14:59:36.835500 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 23 14:59:36.835500 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 23 14:59:36.835500 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Sep 23 14:59:36.835500 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 378/726] 10.222.0.55 () {56 vars in 1096 bytes} [Fri Sep 23 14:59:36 2022] GET /identity/v3/users/swift => generated 83 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 14:59:36.840569 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3285c0c3-27f1-4a54-ab37-bc420062ffe5 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:36.853016 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00501b1-6720-4435-9c23-d46a0fe0e019 None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:36.853385 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00501b1-6720-4435-9c23-d46a0fe0e019 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:36.853671 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00501b1-6720-4435-9c23-d46a0fe0e019 None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:36.855786 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 379/727] 10.222.0.55 () {54 vars in 927 bytes} [Fri Sep 23 14:59:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 23 14:59:36.861975 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290aed80-9768-4097-a484-6fcdd26244b9 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:36.862276 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290aed80-9768-4097-a484-6fcdd26244b9 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:36.862530 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290aed80-9768-4097-a484-6fcdd26244b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:36.872940 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:36.884461 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-290aed80-9768-4097-a484-6fcdd26244b9 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 14:59:36.887021 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-290aed80-9768-4097-a484-6fcdd26244b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 14:59:36.943616 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:36.990404 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 380/728] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Sep 23 14:59:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 23 14:59:36.992110 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feea6781-d76d-4e7f-9b88-4554a0ed3476 None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:36.992550 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feea6781-d76d-4e7f-9b88-4554a0ed3476 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:36.992976 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feea6781-d76d-4e7f-9b88-4554a0ed3476 None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:36.995109 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 381/729] 10.222.0.55 () {54 vars in 927 bytes} [Fri Sep 23 14:59:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 14:59:37.004082 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3285c0c3-27f1-4a54-ab37-bc420062ffe5 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:37.006121 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371009b2-4bab-4c20-a075-8b674e7ea6d3 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:37.006580 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371009b2-4bab-4c20-a075-8b674e7ea6d3 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:37.007009 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371009b2-4bab-4c20-a075-8b674e7ea6d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:37.028010 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-371009b2-4bab-4c20-a075-8b674e7ea6d3 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 14:59:37.029586 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-371009b2-4bab-4c20-a075-8b674e7ea6d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 14:59:37.064779 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4009f41c-d85c-4888-9607-30cd36ce850e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:37.068425 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4009f41c-d85c-4888-9607-30cd36ce850e None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:37.068865 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4009f41c-d85c-4888-9607-30cd36ce850e None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:37.069341 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4009f41c-d85c-4888-9607-30cd36ce850e None admin] PATH_INFO: `/v3/users` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:37.070328 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4009f41c-d85c-4888-9607-30cd36ce850e None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:59:37.071006 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4009f41c-d85c-4888-9607-30cd36ce850e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:37.073998 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4009f41c-d85c-4888-9607-30cd36ce850e None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:37.101834 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 349/730] 10.222.0.55 () {56 vars in 1135 bytes} [Fri Sep 23 14:59:36 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:59:37.109146 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4009f41c-d85c-4888-9607-30cd36ce850e None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:37.136972 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 382/731] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Sep 23 14:59:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 14:59:37.258382 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:37.284699 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4009f41c-d85c-4888-9607-30cd36ce850e None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:37.336458 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb33406e-ac8f-4163-ad3b-f28478dba472 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:37.337590 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb33406e-ac8f-4163-ad3b-f28478dba472 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:37.337866 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb33406e-ac8f-4163-ad3b-f28478dba472 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:37.338217 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb33406e-ac8f-4163-ad3b-f28478dba472 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:37.353148 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371009b2-4bab-4c20-a075-8b674e7ea6d3 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:37.469015 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371009b2-4bab-4c20-a075-8b674e7ea6d3 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:37.472349 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:37.481176 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb33406e-ac8f-4163-ad3b-f28478dba472 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:37.482654 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb33406e-ac8f-4163-ad3b-f28478dba472 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:37.512373 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f8a314-2666-484e-9444-ac26235a581a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:37.513207 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f8a314-2666-484e-9444-ac26235a581a None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:37.513556 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f8a314-2666-484e-9444-ac26235a581a None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:37.513902 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f8a314-2666-484e-9444-ac26235a581a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:37.529176 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f8a314-2666-484e-9444-ac26235a581a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 14:59:37.536580 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-d5f8a314-2666-484e-9444-ac26235a581a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 23 14:59:37.536580 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 14:59:37.536580 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 14:59:37.536580 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 14:59:37.536580 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 14:59:37.536580 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 14:59:37.536580 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 14:59:37.536580 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 14:59:37.536580 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 14:59:37.536580 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 14:59:37.536580 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 14:59:37.536580 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 14:59:37.536580 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 14:59:37.536580 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 14:59:37.536580 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 14:59:37.536580 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 14:59:37.538630 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 14:59:37.538630 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 14:59:37.538630 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 14:59:37.538630 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 14:59:37.538630 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 14:59:37.538630 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 14:59:37.538630 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 14:59:37.538630 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 14:59:37.538630 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 14:59:37.538630 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 14:59:37.538630 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 14:59:37.538630 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Sep 23 14:59:37.538630 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 14:59:37.538630 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Sep 23 14:59:37.538630 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 14:59:37.538630 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 14:59:37.540809 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 14:59:37.540809 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Sep 23 14:59:37.540809 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 14:59:37.540809 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Sep 23 14:59:37.540809 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 14:59:37.540809 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 14:59:37.540809 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 14:59:37.540809 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 14:59:37.540809 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 14:59:37.540809 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 14:59:37.540809 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 23 14:59:37.540809 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Sep 23 14:59:37.548017 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 383/732] 10.222.0.55 () {56 vars in 1096 bytes} [Fri Sep 23 14:59:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 14:59:37.548017 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f8a314-2666-484e-9444-ac26235a581a None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:37.582625 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:37.606393 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 350/733] 10.222.0.55 () {58 vars in 1282 bytes} [Fri Sep 23 14:59:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 498 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 23 14:59:37.607651 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb33406e-ac8f-4163-ad3b-f28478dba472 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:37.663463 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f8a314-2666-484e-9444-ac26235a581a None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:37.700000 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f068670c-474f-4c5b-8cda-356635b40247 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:37.702123 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f068670c-474f-4c5b-8cda-356635b40247 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:37.702123 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f068670c-474f-4c5b-8cda-356635b40247 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:37.702123 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f068670c-474f-4c5b-8cda-356635b40247 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:37.712065 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f068670c-474f-4c5b-8cda-356635b40247 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 14:59:37.717195 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-f068670c-474f-4c5b-8cda-356635b40247 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 23 14:59:37.717195 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 14:59:37.717195 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 14:59:37.717195 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 14:59:37.717195 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 14:59:37.717195 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 14:59:37.717195 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 14:59:37.717195 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 14:59:37.717195 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 14:59:37.717195 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 14:59:37.717195 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 14:59:37.717195 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 14:59:37.717195 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 14:59:37.717195 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 14:59:37.717195 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 14:59:37.717195 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 14:59:37.718160 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 14:59:37.718160 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 14:59:37.718160 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 14:59:37.718160 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 14:59:37.718160 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 14:59:37.718160 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 14:59:37.718160 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 14:59:37.718160 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 14:59:37.718160 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 14:59:37.718160 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 14:59:37.718160 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 14:59:37.718160 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Sep 23 14:59:37.718160 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 14:59:37.718160 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Sep 23 14:59:37.718160 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 14:59:37.718160 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 14:59:37.719069 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 14:59:37.719069 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Sep 23 14:59:37.719069 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 14:59:37.719069 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Sep 23 14:59:37.719069 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 14:59:37.719069 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 14:59:37.719069 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 14:59:37.719069 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 14:59:37.719069 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 14:59:37.719069 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 14:59:37.719069 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 23 14:59:37.719069 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Sep 23 14:59:37.719069 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 384/734] 10.222.0.55 () {56 vars in 1102 bytes} [Fri Sep 23 14:59:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 14:59:37.719069 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f068670c-474f-4c5b-8cda-356635b40247 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:37.725405 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb33406e-ac8f-4163-ad3b-f28478dba472 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:37.740206 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:37.763328 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1ddea7-9a5c-42ed-997d-0f66b6f9df31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:37.763944 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1ddea7-9a5c-42ed-997d-0f66b6f9df31 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:37.764253 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1ddea7-9a5c-42ed-997d-0f66b6f9df31 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:37.764535 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1ddea7-9a5c-42ed-997d-0f66b6f9df31 None admin] PATH_INFO: `/v3/roles` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:37.765200 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1ddea7-9a5c-42ed-997d-0f66b6f9df31 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:59:37.765580 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1ddea7-9a5c-42ed-997d-0f66b6f9df31 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:37.766957 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1ddea7-9a5c-42ed-997d-0f66b6f9df31 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:37.776340 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 351/735] 10.222.0.55 () {56 vars in 1099 bytes} [Fri Sep 23 14:59:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:59:37.777682 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2f66d5-1473-4fbb-808f-4acc4f0e161e None None] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:37.777975 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2f66d5-1473-4fbb-808f-4acc4f0e161e None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:37.778248 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2f66d5-1473-4fbb-808f-4acc4f0e161e None None] PATH_INFO: `/` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:37.779658 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 352/736] 10.222.0.55 () {54 vars in 927 bytes} [Fri Sep 23 14:59:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 14:59:37.780210 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2f66d5-1473-4fbb-808f-4acc4f0e161e None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:37.827638 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:37.837164 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f068670c-474f-4c5b-8cda-356635b40247 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:37.878932 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66605669-b404-4098-a9e6-12e9d095b4dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:37.879551 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66605669-b404-4098-a9e6-12e9d095b4dd None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:37.879833 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66605669-b404-4098-a9e6-12e9d095b4dd None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:37.880131 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66605669-b404-4098-a9e6-12e9d095b4dd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:37.891471 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2f66d5-1473-4fbb-808f-4acc4f0e161e None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:37.891820 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66605669-b404-4098-a9e6-12e9d095b4dd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:37.893280 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66605669-b404-4098-a9e6-12e9d095b4dd None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:37.911259 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-66605669-b404-4098-a9e6-12e9d095b4dd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 14:59:37.911259 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 14:59:37.911259 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 14:59:37.911259 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 14:59:37.911259 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 14:59:37.911259 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 14:59:37.911259 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 14:59:37.911259 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 14:59:37.911259 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 14:59:37.911259 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 14:59:37.911259 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 14:59:37.911259 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 14:59:37.911259 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 14:59:37.911259 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 14:59:37.911259 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 14:59:37.911259 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 14:59:37.912298 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 14:59:37.912298 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 14:59:37.912298 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 14:59:37.912298 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 14:59:37.912298 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 14:59:37.912298 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 14:59:37.912298 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 14:59:37.912298 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Sep 23 14:59:37.912298 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 14:59:37.912298 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Sep 23 14:59:37.912298 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 14:59:37.912298 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 14:59:37.912298 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 14:59:37.912298 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Sep 23 14:59:37.912298 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 14:59:37.912298 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Sep 23 14:59:37.912298 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 14:59:37.913223 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 14:59:37.913223 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 14:59:37.913223 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Sep 23 14:59:37.917560 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 385/737] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Sep 23 14:59:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 14:59:37.918605 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acacb0a4-87e0-4759-8f1a-6c5f687c8be7 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:37.918899 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acacb0a4-87e0-4759-8f1a-6c5f687c8be7 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:37.919168 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acacb0a4-87e0-4759-8f1a-6c5f687c8be7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:37.929878 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b1c4e2-fcca-4019-8e80-74871dcf26e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:37.930495 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b1c4e2-fcca-4019-8e80-74871dcf26e8 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:37.931484 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b1c4e2-fcca-4019-8e80-74871dcf26e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:37.931721 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b1c4e2-fcca-4019-8e80-74871dcf26e8 None admin] PATH_INFO: `/v3/roles` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:37.932612 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b1c4e2-fcca-4019-8e80-74871dcf26e8 None admin] RBAC: Adding query filter params (name=service) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:59:37.932954 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b1c4e2-fcca-4019-8e80-74871dcf26e8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:37.934336 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b1c4e2-fcca-4019-8e80-74871dcf26e8 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:37.937719 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-acacb0a4-87e0-4759-8f1a-6c5f687c8be7 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 14:59:37.938897 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-acacb0a4-87e0-4759-8f1a-6c5f687c8be7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 14:59:37.948690 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 353/738] 10.222.0.55 () {56 vars in 1103 bytes} [Fri Sep 23 14:59:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:59:37.948690 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b1c4e2-fcca-4019-8e80-74871dcf26e8 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:38.004826 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:38.009343 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 386/739] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Sep 23 14:59:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 14:59:38.010304 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acacb0a4-87e0-4759-8f1a-6c5f687c8be7 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:38.059603 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b1c4e2-fcca-4019-8e80-74871dcf26e8 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:38.070263 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:38.099164 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517bc6f3-f63f-4dcf-92d8-21878e7ab854 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:38.100123 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517bc6f3-f63f-4dcf-92d8-21878e7ab854 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:38.100401 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517bc6f3-f63f-4dcf-92d8-21878e7ab854 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:38.100666 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517bc6f3-f63f-4dcf-92d8-21878e7ab854 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:38.112375 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517bc6f3-f63f-4dcf-92d8-21878e7ab854 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:38.113860 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517bc6f3-f63f-4dcf-92d8-21878e7ab854 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:38.115377 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acacb0a4-87e0-4759-8f1a-6c5f687c8be7 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:38.130263 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application [None req-517bc6f3-f63f-4dcf-92d8-21878e7ab854 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 14:59:38.130263 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 14:59:38.130263 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 14:59:38.130263 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 14:59:38.130263 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 14:59:38.130263 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 14:59:38.130263 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 14:59:38.130263 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 14:59:38.130263 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 14:59:38.130263 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 14:59:38.130263 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 14:59:38.130263 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 14:59:38.130263 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 14:59:38.130263 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 14:59:38.130263 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 14:59:38.130263 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 14:59:38.131283 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 14:59:38.131283 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 14:59:38.131283 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 14:59:38.131283 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 14:59:38.131283 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 14:59:38.131283 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 14:59:38.131283 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 14:59:38.131283 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application with Lock( Sep 23 14:59:38.131283 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 14:59:38.131283 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._enter() Sep 23 14:59:38.131283 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 14:59:38.131283 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 14:59:38.131283 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 14:59:38.131283 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.creator() Sep 23 14:59:38.131283 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 14:59:38.131283 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application created_value = creator( Sep 23 14:59:38.131283 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 14:59:38.132214 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 14:59:38.132214 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 14:59:38.132214 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Sep 23 14:59:38.132214 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 354/740] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Sep 23 14:59:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 14:59:38.132214 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517bc6f3-f63f-4dcf-92d8-21878e7ab854 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:38.160159 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4eb7bd9-a0a7-4f2f-bc02-c32031462630 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:38.161041 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4eb7bd9-a0a7-4f2f-bc02-c32031462630 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:38.161327 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4eb7bd9-a0a7-4f2f-bc02-c32031462630 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:38.161920 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4eb7bd9-a0a7-4f2f-bc02-c32031462630 None admin] PATH_INFO: `/v3/domains` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:38.162581 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4eb7bd9-a0a7-4f2f-bc02-c32031462630 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:59:38.162888 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4eb7bd9-a0a7-4f2f-bc02-c32031462630 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:38.164287 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4eb7bd9-a0a7-4f2f-bc02-c32031462630 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:38.176743 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 387/741] 10.222.0.55 () {56 vars in 1109 bytes} [Fri Sep 23 14:59:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:59:38.178115 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c626ff9-d524-498a-9d1a-3bec4e96e4ce None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:38.178401 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c626ff9-d524-498a-9d1a-3bec4e96e4ce None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:38.178671 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c626ff9-d524-498a-9d1a-3bec4e96e4ce None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:38.180131 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 388/742] 10.222.0.55 () {54 vars in 927 bytes} [Fri Sep 23 14:59:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 14:59:38.180712 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c626ff9-d524-498a-9d1a-3bec4e96e4ce None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:38.245599 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517bc6f3-f63f-4dcf-92d8-21878e7ab854 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:38.250301 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:38.282175 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b239d7cc-8647-4233-b66e-6d5150889c2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:38.282994 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b239d7cc-8647-4233-b66e-6d5150889c2d None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:38.283275 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b239d7cc-8647-4233-b66e-6d5150889c2d None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:38.283549 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b239d7cc-8647-4233-b66e-6d5150889c2d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:38.294797 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b239d7cc-8647-4233-b66e-6d5150889c2d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:38.294980 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c626ff9-d524-498a-9d1a-3bec4e96e4ce None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:38.296281 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b239d7cc-8647-4233-b66e-6d5150889c2d None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:38.310424 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application [None req-b239d7cc-8647-4233-b66e-6d5150889c2d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 14:59:38.310424 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 14:59:38.310424 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 14:59:38.310424 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 14:59:38.310424 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 14:59:38.310424 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 14:59:38.310424 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 14:59:38.310424 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 14:59:38.310424 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 14:59:38.310424 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 14:59:38.310424 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 14:59:38.310424 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 14:59:38.310424 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 14:59:38.310424 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 14:59:38.310424 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 14:59:38.310424 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 14:59:38.311504 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 14:59:38.311504 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 14:59:38.311504 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 14:59:38.311504 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 14:59:38.311504 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 14:59:38.311504 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 14:59:38.311504 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 14:59:38.311504 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application with Lock( Sep 23 14:59:38.311504 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 14:59:38.311504 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._enter() Sep 23 14:59:38.311504 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 14:59:38.311504 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 14:59:38.311504 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 14:59:38.311504 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.creator() Sep 23 14:59:38.311504 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 14:59:38.311504 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application created_value = creator( Sep 23 14:59:38.311504 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 14:59:38.312435 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 14:59:38.312435 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 14:59:38.312435 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Sep 23 14:59:38.312435 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 355/743] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Sep 23 14:59:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 14:59:38.312435 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b239d7cc-8647-4233-b66e-6d5150889c2d None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:38.327473 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:38.329393 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea3f8cf-fb05-4ac9-b28a-6d99a5f28237 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:38.329998 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea3f8cf-fb05-4ac9-b28a-6d99a5f28237 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:38.330277 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea3f8cf-fb05-4ac9-b28a-6d99a5f28237 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:38.330545 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea3f8cf-fb05-4ac9-b28a-6d99a5f28237 None admin] PATH_INFO: `/v3/domains` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:38.331180 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea3f8cf-fb05-4ac9-b28a-6d99a5f28237 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:59:38.331476 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea3f8cf-fb05-4ac9-b28a-6d99a5f28237 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:38.332863 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea3f8cf-fb05-4ac9-b28a-6d99a5f28237 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:38.344998 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 389/744] 10.222.0.55 () {56 vars in 1109 bytes} [Fri Sep 23 14:59:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:59:38.346263 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e6bcb1-8efa-4bbb-bfca-f72143b02569 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:38.346545 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e6bcb1-8efa-4bbb-bfca-f72143b02569 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:38.346821 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e6bcb1-8efa-4bbb-bfca-f72143b02569 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:38.364123 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-44e6bcb1-8efa-4bbb-bfca-f72143b02569 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 14:59:38.365308 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-44e6bcb1-8efa-4bbb-bfca-f72143b02569 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 14:59:38.418354 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b239d7cc-8647-4233-b66e-6d5150889c2d None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:38.432344 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 390/745] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Sep 23 14:59:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 14:59:38.433170 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e6bcb1-8efa-4bbb-bfca-f72143b02569 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:38.457627 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea65aa25-b776-4edf-9252-dd8f03ed86ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:38.458271 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea65aa25-b776-4edf-9252-dd8f03ed86ce None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:38.458554 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea65aa25-b776-4edf-9252-dd8f03ed86ce None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:38.458826 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea65aa25-b776-4edf-9252-dd8f03ed86ce None admin] PATH_INFO: `/v3/projects/service` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:38.466358 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea65aa25-b776-4edf-9252-dd8f03ed86ce None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:38.467864 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea65aa25-b776-4edf-9252-dd8f03ed86ce None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:38.479340 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application [None req-ea65aa25-b776-4edf-9252-dd8f03ed86ce None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 14:59:38.479340 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 14:59:38.479340 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 14:59:38.479340 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 14:59:38.479340 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 14:59:38.479340 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 14:59:38.479340 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 14:59:38.479340 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 14:59:38.479340 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 14:59:38.479340 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 14:59:38.479340 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 14:59:38.479340 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 14:59:38.479340 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 23 14:59:38.479340 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 23 14:59:38.479340 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 23 14:59:38.480310 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 23 14:59:38.480310 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 14:59:38.480310 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 14:59:38.480310 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 14:59:38.480310 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 14:59:38.480310 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 14:59:38.480310 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 14:59:38.480310 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 14:59:38.480310 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application with Lock( Sep 23 14:59:38.480310 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 14:59:38.480310 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._enter() Sep 23 14:59:38.480310 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 14:59:38.480310 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 14:59:38.480310 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 14:59:38.480310 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.creator() Sep 23 14:59:38.480310 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 14:59:38.480310 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application created_value = creator( Sep 23 14:59:38.481285 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 23 14:59:38.481285 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 23 14:59:38.481285 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 23 14:59:38.481285 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 23 14:59:38.481285 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 23 14:59:38.481285 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 23 14:59:38.481285 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 14:59:38.481285 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Sep 23 14:59:38.481285 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 356/746] 10.222.0.55 () {56 vars in 1111 bytes} [Fri Sep 23 14:59:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 14:59:38.481285 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea65aa25-b776-4edf-9252-dd8f03ed86ce None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:38.495108 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:38.551840 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e6bcb1-8efa-4bbb-bfca-f72143b02569 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:38.556420 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:38.592400 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948757c5-cde6-4e9c-8431-902b0a8cbbb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:38.593074 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948757c5-cde6-4e9c-8431-902b0a8cbbb4 None admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:38.593314 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948757c5-cde6-4e9c-8431-902b0a8cbbb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:38.593587 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948757c5-cde6-4e9c-8431-902b0a8cbbb4 None admin] PATH_INFO: `/v3/services` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:38.594239 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948757c5-cde6-4e9c-8431-902b0a8cbbb4 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:38.594855 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea65aa25-b776-4edf-9252-dd8f03ed86ce None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:38.595618 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948757c5-cde6-4e9c-8431-902b0a8cbbb4 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:38.628765 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2689817-1489-4a5f-9d2c-351426e8e068 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:38.629378 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2689817-1489-4a5f-9d2c-351426e8e068 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:38.629662 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2689817-1489-4a5f-9d2c-351426e8e068 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:38.629936 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2689817-1489-4a5f-9d2c-351426e8e068 None admin] PATH_INFO: `/v3/domains` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:38.630570 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2689817-1489-4a5f-9d2c-351426e8e068 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:59:38.630876 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2689817-1489-4a5f-9d2c-351426e8e068 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:38.632335 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2689817-1489-4a5f-9d2c-351426e8e068 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:38.645782 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 357/747] 10.222.0.55 () {56 vars in 1109 bytes} [Fri Sep 23 14:59:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:59:38.646579 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2689817-1489-4a5f-9d2c-351426e8e068 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:38.706582 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 391/748] 10.222.0.55 () {60 vars in 1141 bytes} [Fri Sep 23 14:59:38 2022] POST /identity/v3/services => generated 240 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 14:59:38.718195 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948757c5-cde6-4e9c-8431-902b0a8cbbb4 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:38.757491 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2689817-1489-4a5f-9d2c-351426e8e068 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:38.790663 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:38.807541 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb637aa7-f661-42ba-9ff7-37e4684e0a1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:38.808152 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb637aa7-f661-42ba-9ff7-37e4684e0a1d None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:38.808435 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb637aa7-f661-42ba-9ff7-37e4684e0a1d None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:38.808709 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb637aa7-f661-42ba-9ff7-37e4684e0a1d None admin] PATH_INFO: `/v3/users/nova` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:38.817699 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb637aa7-f661-42ba-9ff7-37e4684e0a1d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:38.819244 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb637aa7-f661-42ba-9ff7-37e4684e0a1d None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:38.837174 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application [None req-eb637aa7-f661-42ba-9ff7-37e4684e0a1d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 23 14:59:38.837174 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 14:59:38.837174 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 14:59:38.837174 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 14:59:38.837174 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 14:59:38.837174 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 14:59:38.837174 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 14:59:38.837174 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 14:59:38.837174 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 14:59:38.837174 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 14:59:38.837174 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 14:59:38.837174 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 14:59:38.837174 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 23 14:59:38.837174 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 23 14:59:38.837174 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 23 14:59:38.837174 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 23 14:59:38.838311 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 14:59:38.838311 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 14:59:38.838311 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 23 14:59:38.838311 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 14:59:38.838311 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 23 14:59:38.838311 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 14:59:38.838311 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 14:59:38.838311 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 14:59:38.838311 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 14:59:38.838311 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 14:59:38.838311 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 14:59:38.838311 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application with Lock( Sep 23 14:59:38.838311 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 14:59:38.838311 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._enter() Sep 23 14:59:38.838311 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 14:59:38.838311 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 14:59:38.838311 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 14:59:38.838311 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.creator() Sep 23 14:59:38.839257 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 14:59:38.839257 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application created_value = creator( Sep 23 14:59:38.839257 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 23 14:59:38.839257 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 23 14:59:38.839257 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 23 14:59:38.839257 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 23 14:59:38.839257 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 23 14:59:38.839257 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 23 14:59:38.839257 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 23 14:59:38.839257 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Sep 23 14:59:38.839257 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 358/749] 10.222.0.55 () {56 vars in 1093 bytes} [Fri Sep 23 14:59:38 2022] GET /identity/v3/users/nova => generated 82 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 14:59:38.839854 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb637aa7-f661-42ba-9ff7-37e4684e0a1d None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:38.843505 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948757c5-cde6-4e9c-8431-902b0a8cbbb4 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:38.857332 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:38.886641 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a899b131-1a22-4ce3-95ea-6ec44c183f60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:38.887641 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a899b131-1a22-4ce3-95ea-6ec44c183f60 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:38.887641 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a899b131-1a22-4ce3-95ea-6ec44c183f60 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:38.887904 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a899b131-1a22-4ce3-95ea-6ec44c183f60 None admin] PATH_INFO: `/v3/projects` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:38.888749 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a899b131-1a22-4ce3-95ea-6ec44c183f60 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:59:38.889059 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a899b131-1a22-4ce3-95ea-6ec44c183f60 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:38.890505 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a899b131-1a22-4ce3-95ea-6ec44c183f60 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:38.905609 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 392/750] 10.222.0.55 () {56 vars in 1148 bytes} [Fri Sep 23 14:59:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:59:38.907223 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a899b131-1a22-4ce3-95ea-6ec44c183f60 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:38.961335 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb637aa7-f661-42ba-9ff7-37e4684e0a1d None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:38.996528 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cb06f7-599e-491d-8a98-da0154cc658f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:38.997169 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cb06f7-599e-491d-8a98-da0154cc658f None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:38.997469 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cb06f7-599e-491d-8a98-da0154cc658f None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:38.997734 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cb06f7-599e-491d-8a98-da0154cc658f None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:39.005668 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cb06f7-599e-491d-8a98-da0154cc658f None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:39.007130 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cb06f7-599e-491d-8a98-da0154cc658f None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:39.019029 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application [None req-d5cb06f7-599e-491d-8a98-da0154cc658f None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 23 14:59:39.019029 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 14:59:39.019029 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 14:59:39.019029 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 14:59:39.019029 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 14:59:39.019029 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 14:59:39.019029 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 14:59:39.019029 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 14:59:39.019029 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 14:59:39.019029 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 14:59:39.019029 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 14:59:39.019029 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 14:59:39.019029 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 23 14:59:39.019029 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 23 14:59:39.019029 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 23 14:59:39.019029 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 23 14:59:39.020017 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 14:59:39.020017 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 14:59:39.020017 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 23 14:59:39.020017 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 14:59:39.020017 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 23 14:59:39.020017 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 14:59:39.020017 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 14:59:39.020017 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 14:59:39.020017 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 14:59:39.020017 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 14:59:39.020017 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 14:59:39.020017 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application with Lock( Sep 23 14:59:39.020017 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 14:59:39.020017 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._enter() Sep 23 14:59:39.020017 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 14:59:39.020017 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 14:59:39.020017 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 14:59:39.020017 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.creator() Sep 23 14:59:39.020974 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 14:59:39.020974 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application created_value = creator( Sep 23 14:59:39.020974 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 23 14:59:39.020974 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 23 14:59:39.020974 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 23 14:59:39.020974 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 23 14:59:39.020974 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 23 14:59:39.020974 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 23 14:59:39.020974 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 23 14:59:39.020974 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Sep 23 14:59:39.020974 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 359/751] 10.222.0.55 () {56 vars in 1117 bytes} [Fri Sep 23 14:59:38 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 14:59:39.020974 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cb06f7-599e-491d-8a98-da0154cc658f None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:39.030356 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a899b131-1a22-4ce3-95ea-6ec44c183f60 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:39.074622 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4a20fa-c292-4e16-a0d1-01740797f74d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:39.075338 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4a20fa-c292-4e16-a0d1-01740797f74d None admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:39.075624 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4a20fa-c292-4e16-a0d1-01740797f74d None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:39.078421 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4a20fa-c292-4e16-a0d1-01740797f74d None admin] PATH_INFO: `/v3/services` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:39.078421 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4a20fa-c292-4e16-a0d1-01740797f74d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:39.078966 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4a20fa-c292-4e16-a0d1-01740797f74d None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:39.080510 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:39.134206 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cb06f7-599e-491d-8a98-da0154cc658f None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:39.171408 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d44aa9-d366-449b-b5f2-d1b1d14d2162 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:39.172046 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d44aa9-d366-449b-b5f2-d1b1d14d2162 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:39.172327 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d44aa9-d366-449b-b5f2-d1b1d14d2162 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:39.176979 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d44aa9-d366-449b-b5f2-d1b1d14d2162 None admin] PATH_INFO: `/v3/users` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:39.176979 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d44aa9-d366-449b-b5f2-d1b1d14d2162 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:59:39.176979 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d44aa9-d366-449b-b5f2-d1b1d14d2162 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:39.176979 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d44aa9-d366-449b-b5f2-d1b1d14d2162 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:39.192816 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 360/752] 10.222.0.55 () {56 vars in 1133 bytes} [Fri Sep 23 14:59:39 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:59:39.193456 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d44aa9-d366-449b-b5f2-d1b1d14d2162 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:39.216057 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 393/753] 10.222.0.55 () {60 vars in 1141 bytes} [Fri Sep 23 14:59:38 2022] POST /identity/v3/services => generated 251 bytes in 306 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 14:59:39.216057 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4a20fa-c292-4e16-a0d1-01740797f74d None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:39.252793 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:39.348016 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d44aa9-d366-449b-b5f2-d1b1d14d2162 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:39.350359 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:39.355150 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4a20fa-c292-4e16-a0d1-01740797f74d None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:39.395267 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9528d3e0-0624-498f-b49f-6cb99af6e541 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:39.396254 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9528d3e0-0624-498f-b49f-6cb99af6e541 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:39.396722 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9528d3e0-0624-498f-b49f-6cb99af6e541 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:39.397212 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9528d3e0-0624-498f-b49f-6cb99af6e541 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:39.398129 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9528d3e0-0624-498f-b49f-6cb99af6e541 None admin] RBAC: Adding query filter params (role.id=8e2919e79df040e8926f21607818b58c, scope.project.id=87b3e9d4b6474bd4a04ba59ef52bf3e2, user.id=2228c0b49a0249bbaf524a8c4e9e108b) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:59:39.398622 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9528d3e0-0624-498f-b49f-6cb99af6e541 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:39.400283 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9528d3e0-0624-498f-b49f-6cb99af6e541 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:39.404829 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6142e306-a203-45fe-9baa-09027231ca7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:39.405510 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6142e306-a203-45fe-9baa-09027231ca7d None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:39.405796 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6142e306-a203-45fe-9baa-09027231ca7d None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:39.406068 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6142e306-a203-45fe-9baa-09027231ca7d None admin] PATH_INFO: `/v3/users` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:39.406804 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6142e306-a203-45fe-9baa-09027231ca7d None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:59:39.407113 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6142e306-a203-45fe-9baa-09027231ca7d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:39.408892 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6142e306-a203-45fe-9baa-09027231ca7d None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:39.409266 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 361/754] 10.222.0.55 () {56 vars in 1374 bytes} [Fri Sep 23 14:59:39 2022] GET /identity/v3/role_assignments?user.id=2228c0b49a0249bbaf524a8c4e9e108b&scope.project.id=87b3e9d4b6474bd4a04ba59ef52bf3e2&role.id=8e2919e79df040e8926f21607818b58c => generated 259 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:59:39.410412 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9528d3e0-0624-498f-b49f-6cb99af6e541 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:39.437863 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 394/755] 10.222.0.55 () {56 vars in 1149 bytes} [Fri Sep 23 14:59:39 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:59:39.440008 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6142e306-a203-45fe-9baa-09027231ca7d None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:39.563643 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9528d3e0-0624-498f-b49f-6cb99af6e541 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:39.565130 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:39.589168 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6142e306-a203-45fe-9baa-09027231ca7d None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:39.607326 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7feb04ff-e1c9-4039-8805-bc1f2a8bb1dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:39.608012 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7feb04ff-e1c9-4039-8805-bc1f2a8bb1dd None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:39.608244 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7feb04ff-e1c9-4039-8805-bc1f2a8bb1dd None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:39.608525 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7feb04ff-e1c9-4039-8805-bc1f2a8bb1dd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:39.623877 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7feb04ff-e1c9-4039-8805-bc1f2a8bb1dd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:39.625590 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7feb04ff-e1c9-4039-8805-bc1f2a8bb1dd None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:39.638655 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64728fc-ffbf-4d03-bed2-fb191c565947 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:39.639624 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64728fc-ffbf-4d03-bed2-fb191c565947 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:39.640143 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64728fc-ffbf-4d03-bed2-fb191c565947 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:39.640621 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64728fc-ffbf-4d03-bed2-fb191c565947 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:39.643205 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application [None req-7feb04ff-e1c9-4039-8805-bc1f2a8bb1dd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 14:59:39.643205 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 14:59:39.643205 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 14:59:39.643205 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 14:59:39.643205 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 14:59:39.643205 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 14:59:39.643205 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 14:59:39.643205 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 14:59:39.643205 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 14:59:39.643205 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 14:59:39.643205 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 14:59:39.643205 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 14:59:39.643205 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 14:59:39.643205 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 14:59:39.643205 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 14:59:39.643205 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 14:59:39.644268 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 14:59:39.644268 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 14:59:39.644268 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 14:59:39.644268 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 14:59:39.644268 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 14:59:39.644268 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 14:59:39.644268 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 14:59:39.644268 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application with Lock( Sep 23 14:59:39.644268 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 14:59:39.644268 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._enter() Sep 23 14:59:39.644268 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 14:59:39.644268 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 14:59:39.644268 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 14:59:39.644268 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.creator() Sep 23 14:59:39.644268 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 14:59:39.644268 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application created_value = creator( Sep 23 14:59:39.644268 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 14:59:39.645201 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 14:59:39.645201 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 14:59:39.645201 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Sep 23 14:59:39.645851 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 362/756] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Sep 23 14:59:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 14:59:39.650928 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7feb04ff-e1c9-4039-8805-bc1f2a8bb1dd None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:39.672897 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:39.760993 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64728fc-ffbf-4d03-bed2-fb191c565947 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:39.762703 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64728fc-ffbf-4d03-bed2-fb191c565947 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:39.810806 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7feb04ff-e1c9-4039-8805-bc1f2a8bb1dd None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:39.864093 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66817232-bcae-4c02-bc1f-27452203fddd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:39.864628 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66817232-bcae-4c02-bc1f-27452203fddd None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:39.864910 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66817232-bcae-4c02-bc1f-27452203fddd None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:39.868074 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66817232-bcae-4c02-bc1f-27452203fddd None admin] PATH_INFO: `/v3/domains` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:39.868788 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66817232-bcae-4c02-bc1f-27452203fddd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:59:39.869105 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66817232-bcae-4c02-bc1f-27452203fddd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:39.870483 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66817232-bcae-4c02-bc1f-27452203fddd None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:39.899676 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 363/757] 10.222.0.55 () {56 vars in 1109 bytes} [Fri Sep 23 14:59:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 14:59:39.907313 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66817232-bcae-4c02-bc1f-27452203fddd None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:39.953825 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:39.992531 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 395/758] 10.222.0.55 () {58 vars in 1282 bytes} [Fri Sep 23 14:59:39 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 553 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 23 14:59:39.999140 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64728fc-ffbf-4d03-bed2-fb191c565947 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:40.005658 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:40.031195 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66817232-bcae-4c02-bc1f-27452203fddd None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:40.098665 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23c0e44-23b1-4aee-8ad8-9c706b376b57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:40.099315 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23c0e44-23b1-4aee-8ad8-9c706b376b57 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:40.099591 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23c0e44-23b1-4aee-8ad8-9c706b376b57 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:40.099865 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23c0e44-23b1-4aee-8ad8-9c706b376b57 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:40.117221 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23c0e44-23b1-4aee-8ad8-9c706b376b57 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:40.119236 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23c0e44-23b1-4aee-8ad8-9c706b376b57 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:40.144256 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application [None req-e23c0e44-23b1-4aee-8ad8-9c706b376b57 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 14:59:40.144256 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 14:59:40.144256 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 14:59:40.144256 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 14:59:40.144256 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 14:59:40.144256 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 14:59:40.144256 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 14:59:40.144256 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 14:59:40.144256 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 14:59:40.144256 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 14:59:40.144256 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 14:59:40.144256 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 14:59:40.144256 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 23 14:59:40.144256 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 23 14:59:40.144256 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 23 14:59:40.145219 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 23 14:59:40.145219 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 14:59:40.145219 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 14:59:40.145219 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 14:59:40.145219 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 14:59:40.145219 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 14:59:40.145219 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 14:59:40.145219 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 14:59:40.145219 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application with Lock( Sep 23 14:59:40.145219 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 14:59:40.145219 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._enter() Sep 23 14:59:40.145219 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 14:59:40.145219 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 14:59:40.145219 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 14:59:40.145219 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.creator() Sep 23 14:59:40.145219 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 14:59:40.145219 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application created_value = creator( Sep 23 14:59:40.146716 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 23 14:59:40.146716 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 23 14:59:40.146716 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 23 14:59:40.146716 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 23 14:59:40.146716 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 23 14:59:40.146716 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 23 14:59:40.146716 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 14:59:40.146716 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Sep 23 14:59:40.146716 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 364/759] 10.222.0.55 () {56 vars in 1111 bytes} [Fri Sep 23 14:59:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 14:59:40.156252 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23c0e44-23b1-4aee-8ad8-9c706b376b57 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:40.204264 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64728fc-ffbf-4d03-bed2-fb191c565947 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:40.244874 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ca27c8-2e00-47c1-b425-058f5d035677 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:40.245518 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ca27c8-2e00-47c1-b425-058f5d035677 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:40.245937 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ca27c8-2e00-47c1-b425-058f5d035677 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:40.246176 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ca27c8-2e00-47c1-b425-058f5d035677 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:40.257748 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ca27c8-2e00-47c1-b425-058f5d035677 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:40.260019 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ca27c8-2e00-47c1-b425-058f5d035677 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:40.277054 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-25ca27c8-2e00-47c1-b425-058f5d035677 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 14:59:40.277054 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 14:59:40.277054 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 14:59:40.277054 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 14:59:40.277054 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 14:59:40.277054 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 14:59:40.277054 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 14:59:40.277054 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 14:59:40.277054 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 14:59:40.277054 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 14:59:40.277054 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 14:59:40.277054 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 14:59:40.277054 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 14:59:40.277054 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 14:59:40.277054 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 14:59:40.277054 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 14:59:40.279378 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 14:59:40.279378 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 14:59:40.279378 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 14:59:40.279378 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 14:59:40.279378 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 14:59:40.279378 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 14:59:40.279378 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 14:59:40.279378 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Sep 23 14:59:40.279378 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 14:59:40.279378 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Sep 23 14:59:40.279378 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 14:59:40.279378 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 14:59:40.279378 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 14:59:40.279378 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Sep 23 14:59:40.279378 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 14:59:40.279378 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Sep 23 14:59:40.279378 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 14:59:40.281610 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 14:59:40.281610 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 14:59:40.281610 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Sep 23 14:59:40.281610 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 396/760] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Sep 23 14:59:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 23 14:59:40.288205 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ca27c8-2e00-47c1-b425-058f5d035677 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:40.334716 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:40.392893 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23c0e44-23b1-4aee-8ad8-9c706b376b57 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:40.412155 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ca27c8-2e00-47c1-b425-058f5d035677 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:40.452215 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7373e7d-22da-4164-b7ef-bbb14a066bf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:40.453207 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7373e7d-22da-4164-b7ef-bbb14a066bf9 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:40.453697 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7373e7d-22da-4164-b7ef-bbb14a066bf9 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:40.454174 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7373e7d-22da-4164-b7ef-bbb14a066bf9 None admin] PATH_INFO: `/v3/domains` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:40.455043 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7373e7d-22da-4164-b7ef-bbb14a066bf9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:59:40.455573 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7373e7d-22da-4164-b7ef-bbb14a066bf9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:40.457278 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7373e7d-22da-4164-b7ef-bbb14a066bf9 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:40.466058 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b20a75b-39ef-4c71-9923-a96539a83b85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:40.466665 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b20a75b-39ef-4c71-9923-a96539a83b85 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:40.466946 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b20a75b-39ef-4c71-9923-a96539a83b85 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:40.467259 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b20a75b-39ef-4c71-9923-a96539a83b85 None admin] PATH_INFO: `/v3/projects` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:40.467955 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b20a75b-39ef-4c71-9923-a96539a83b85 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:59:40.468290 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b20a75b-39ef-4c71-9923-a96539a83b85 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:40.469751 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b20a75b-39ef-4c71-9923-a96539a83b85 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:40.482566 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:40.486004 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 397/761] 10.222.0.55 () {56 vars in 1109 bytes} [Fri Sep 23 14:59:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 14:59:40.489373 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 365/762] 10.222.0.55 () {56 vars in 1148 bytes} [Fri Sep 23 14:59:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:59:40.495070 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7373e7d-22da-4164-b7ef-bbb14a066bf9 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:40.495419 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b20a75b-39ef-4c71-9923-a96539a83b85 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:40.648422 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:40.658516 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7373e7d-22da-4164-b7ef-bbb14a066bf9 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:40.721019 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b20a75b-39ef-4c71-9923-a96539a83b85 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:40.721747 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a86a8f8-599e-4341-9871-40a0f9664842 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:40.722783 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a86a8f8-599e-4341-9871-40a0f9664842 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:40.723247 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a86a8f8-599e-4341-9871-40a0f9664842 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:40.723704 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a86a8f8-599e-4341-9871-40a0f9664842 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:40.742520 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a86a8f8-599e-4341-9871-40a0f9664842 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:40.747505 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a86a8f8-599e-4341-9871-40a0f9664842 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:40.766191 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-1a86a8f8-599e-4341-9871-40a0f9664842 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 14:59:40.766191 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 14:59:40.766191 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 14:59:40.766191 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 14:59:40.766191 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 14:59:40.766191 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 14:59:40.766191 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 14:59:40.766191 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 14:59:40.766191 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 14:59:40.766191 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 14:59:40.766191 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 14:59:40.766191 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 14:59:40.766191 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 23 14:59:40.766191 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 23 14:59:40.766191 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 23 14:59:40.767142 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 23 14:59:40.767142 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 14:59:40.767142 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 14:59:40.767142 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 14:59:40.767142 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 14:59:40.767142 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 14:59:40.767142 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 14:59:40.767142 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 14:59:40.767142 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Sep 23 14:59:40.767142 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 14:59:40.767142 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Sep 23 14:59:40.767142 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 14:59:40.767142 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 14:59:40.767142 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 14:59:40.767142 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Sep 23 14:59:40.767142 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 14:59:40.767142 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Sep 23 14:59:40.768068 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 23 14:59:40.768068 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 23 14:59:40.768068 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 23 14:59:40.768068 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 23 14:59:40.768068 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 23 14:59:40.768068 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 23 14:59:40.768068 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 14:59:40.768068 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Sep 23 14:59:40.779083 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 398/763] 10.222.0.55 () {56 vars in 1111 bytes} [Fri Sep 23 14:59:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 14:59:40.784058 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a86a8f8-599e-4341-9871-40a0f9664842 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:40.820705 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac0bc88-91da-4372-a551-74d30cd2153c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:40.821522 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac0bc88-91da-4372-a551-74d30cd2153c None admin] REQUEST_METHOD: `PUT` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:40.821863 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac0bc88-91da-4372-a551-74d30cd2153c None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:40.822139 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac0bc88-91da-4372-a551-74d30cd2153c None admin] PATH_INFO: `/v3/projects/87b3e9d4b6474bd4a04ba59ef52bf3e2/users/b3c6c7a1325a47be924268e1aa1a7a18/roles/8e2919e79df040e8926f21607818b58c` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:40.828769 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac0bc88-91da-4372-a551-74d30cd2153c None admin] RBAC: Authorizing `identity:create_grant(project_id=87b3e9d4b6474bd4a04ba59ef52bf3e2, user_id=b3c6c7a1325a47be924268e1aa1a7a18, role_id=8e2919e79df040e8926f21607818b58c)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:40.830219 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac0bc88-91da-4372-a551-74d30cd2153c None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:40.905891 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a86a8f8-599e-4341-9871-40a0f9664842 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:40.943018 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 366/764] 10.222.0.55 () {58 vars in 1439 bytes} [Fri Sep 23 14:59:40 2022] PUT /identity/v3/projects/87b3e9d4b6474bd4a04ba59ef52bf3e2/users/b3c6c7a1325a47be924268e1aa1a7a18/roles/8e2919e79df040e8926f21607818b58c => generated 0 bytes in 448 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 14:59:40.950278 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef524f1-7d60-4162-8f8a-14ca0bd5af83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:40.950969 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef524f1-7d60-4162-8f8a-14ca0bd5af83 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:40.951255 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef524f1-7d60-4162-8f8a-14ca0bd5af83 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:40.951530 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef524f1-7d60-4162-8f8a-14ca0bd5af83 None admin] PATH_INFO: `/v3/projects` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:40.952277 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef524f1-7d60-4162-8f8a-14ca0bd5af83 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:59:40.952588 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef524f1-7d60-4162-8f8a-14ca0bd5af83 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:40.954086 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef524f1-7d60-4162-8f8a-14ca0bd5af83 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:40.979498 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 399/765] 10.222.0.55 () {56 vars in 1148 bytes} [Fri Sep 23 14:59:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:59:40.982398 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac0bc88-91da-4372-a551-74d30cd2153c None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:41.107647 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:41.136565 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbdcfe2-3f1c-4a13-b311-88935fd963a3 None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:41.136918 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbdcfe2-3f1c-4a13-b311-88935fd963a3 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:41.137194 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbdcfe2-3f1c-4a13-b311-88935fd963a3 None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:41.138929 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 400/766] 10.222.0.55 () {54 vars in 927 bytes} [Fri Sep 23 14:59:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 23 14:59:41.154156 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d468b1d7-6ea8-4d62-89c5-eed923be8e52 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:41.154472 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d468b1d7-6ea8-4d62-89c5-eed923be8e52 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:41.154706 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d468b1d7-6ea8-4d62-89c5-eed923be8e52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:41.168208 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac0bc88-91da-4372-a551-74d30cd2153c None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:41.193080 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-d468b1d7-6ea8-4d62-89c5-eed923be8e52 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 14:59:41.194301 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-d468b1d7-6ea8-4d62-89c5-eed923be8e52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 14:59:41.216087 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:41.226546 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c3c395-32d6-4786-98e4-95fde78cc2bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:41.236828 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c3c395-32d6-4786-98e4-95fde78cc2bb None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:41.236828 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c3c395-32d6-4786-98e4-95fde78cc2bb None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:41.236828 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c3c395-32d6-4786-98e4-95fde78cc2bb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:41.236828 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c3c395-32d6-4786-98e4-95fde78cc2bb None admin] RBAC: Adding query filter params (role.id=0665869b71fa40b0b552efdd57faf023, scope.project.id=87b3e9d4b6474bd4a04ba59ef52bf3e2, user.id=68206a6b4f244d45bf7bf1504396e430) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:59:41.236828 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c3c395-32d6-4786-98e4-95fde78cc2bb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:41.236828 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c3c395-32d6-4786-98e4-95fde78cc2bb None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:41.240933 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 367/767] 10.222.0.55 () {56 vars in 1374 bytes} [Fri Sep 23 14:59:40 2022] GET /identity/v3/role_assignments?user.id=68206a6b4f244d45bf7bf1504396e430&scope.project.id=87b3e9d4b6474bd4a04ba59ef52bf3e2&role.id=0665869b71fa40b0b552efdd57faf023 => generated 259 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:59:41.297954 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 401/768] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Sep 23 14:59:41 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 23 14:59:41.588462 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c3c395-32d6-4786-98e4-95fde78cc2bb None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:41.712766 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c3c395-32d6-4786-98e4-95fde78cc2bb None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:41.744844 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:41.763105 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a355a25-a68e-4c36-b989-7b601eaf2a72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:41.764123 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a355a25-a68e-4c36-b989-7b601eaf2a72 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:41.764570 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a355a25-a68e-4c36-b989-7b601eaf2a72 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:41.768794 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a355a25-a68e-4c36-b989-7b601eaf2a72 None admin] PATH_INFO: `/v3/services/network` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:41.768794 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a355a25-a68e-4c36-b989-7b601eaf2a72 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:41.768794 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a355a25-a68e-4c36-b989-7b601eaf2a72 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:41.778006 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application [None req-4a355a25-a68e-4c36-b989-7b601eaf2a72 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 23 14:59:41.778006 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 14:59:41.778006 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 23 14:59:41.778006 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 23 14:59:41.778006 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 23 14:59:41.778006 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 23 14:59:41.778006 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 23 14:59:41.778006 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 23 14:59:41.778006 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 23 14:59:41.778006 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Sep 23 14:59:41.778006 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 23 14:59:41.778006 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Sep 23 14:59:41.778006 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 14:59:41.778006 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 14:59:41.778006 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 14:59:41.778006 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 14:59:41.778006 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 14:59:41.779044 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 14:59:41.779044 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 14:59:41.779044 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 14:59:41.779044 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 14:59:41.779044 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 14:59:41.779044 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 14:59:41.779044 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 23 14:59:41.779044 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 23 14:59:41.779044 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 23 14:59:41.779044 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 23 14:59:41.779044 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 14:59:41.779044 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 14:59:41.779044 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 14:59:41.779044 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 14:59:41.779044 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 14:59:41.779044 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 14:59:41.779875 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 14:59:41.779875 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application with Lock( Sep 23 14:59:41.779875 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 14:59:41.779875 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._enter() Sep 23 14:59:41.779875 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 14:59:41.779875 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 14:59:41.779875 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 14:59:41.779875 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.creator() Sep 23 14:59:41.779875 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 14:59:41.779875 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application created_value = creator( Sep 23 14:59:41.779875 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 23 14:59:41.779875 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 23 14:59:41.779875 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 23 14:59:41.779875 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Sep 23 14:59:41.783106 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 368/769] 10.222.0.55 () {56 vars in 1111 bytes} [Fri Sep 23 14:59:41 2022] GET /identity/v3/services/network => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 14:59:41.789297 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d468b1d7-6ea8-4d62-89c5-eed923be8e52 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:41.931461 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d468b1d7-6ea8-4d62-89c5-eed923be8e52 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:41.986110 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8b8fbc-c529-43f4-8b7b-8ba5b962820a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:41.986735 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8b8fbc-c529-43f4-8b7b-8ba5b962820a None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:41.987018 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8b8fbc-c529-43f4-8b7b-8ba5b962820a None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:41.987296 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8b8fbc-c529-43f4-8b7b-8ba5b962820a None admin] PATH_INFO: `/v3/services` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:41.987933 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8b8fbc-c529-43f4-8b7b-8ba5b962820a None admin] RBAC: Adding query filter params (name=network) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:59:41.988267 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8b8fbc-c529-43f4-8b7b-8ba5b962820a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:41.989627 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8b8fbc-c529-43f4-8b7b-8ba5b962820a None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:42.000400 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 402/770] 10.222.0.55 () {56 vars in 1112 bytes} [Fri Sep 23 14:59:41 2022] GET /identity/v3/services?name=network => generated 124 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:59:42.005946 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a355a25-a68e-4c36-b989-7b601eaf2a72 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:42.010326 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1469ab-e14e-4986-8b3a-0cf555501bf1 None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:42.010599 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1469ab-e14e-4986-8b3a-0cf555501bf1 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:42.010877 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1469ab-e14e-4986-8b3a-0cf555501bf1 None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:42.012602 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 403/771] 10.222.0.55 () {54 vars in 927 bytes} [Fri Sep 23 14:59:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 14:59:42.017756 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b74a7a9-0780-4245-893a-24210f554661 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:42.018041 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b74a7a9-0780-4245-893a-24210f554661 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:42.018359 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b74a7a9-0780-4245-893a-24210f554661 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:42.050890 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-6b74a7a9-0780-4245-893a-24210f554661 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 14:59:42.052962 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-6b74a7a9-0780-4245-893a-24210f554661 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 14:59:42.059004 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:42.155474 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a355a25-a68e-4c36-b989-7b601eaf2a72 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:42.191891 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80749968-3dd3-4036-8081-704125d47ae1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:42.192981 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80749968-3dd3-4036-8081-704125d47ae1 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:42.193490 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80749968-3dd3-4036-8081-704125d47ae1 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:42.193964 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80749968-3dd3-4036-8081-704125d47ae1 None admin] PATH_INFO: `/v3/services` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:42.194796 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80749968-3dd3-4036-8081-704125d47ae1 None admin] RBAC: Adding query filter params (type=network) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:59:42.195379 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80749968-3dd3-4036-8081-704125d47ae1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:42.197559 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80749968-3dd3-4036-8081-704125d47ae1 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:42.203478 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 369/772] 10.222.0.55 () {56 vars in 1112 bytes} [Fri Sep 23 14:59:42 2022] GET /identity/v3/services?type=network => generated 350 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:59:42.209686 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80749968-3dd3-4036-8081-704125d47ae1 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:42.250220 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 404/773] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Sep 23 14:59:42 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 23 14:59:42.268449 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9eae4b9-f21f-4685-bea8-854b46e85664 None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:42.268744 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9eae4b9-f21f-4685-bea8-854b46e85664 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:42.268992 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9eae4b9-f21f-4685-bea8-854b46e85664 None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:42.270349 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 405/774] 10.222.0.55 () {54 vars in 927 bytes} [Fri Sep 23 14:59:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 23 14:59:42.270820 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:42.276114 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e809c1f5-fd4e-4f0f-aa70-a8cc60a1ade5 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:42.276399 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e809c1f5-fd4e-4f0f-aa70-a8cc60a1ade5 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:42.276675 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e809c1f5-fd4e-4f0f-aa70-a8cc60a1ade5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:42.297384 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-e809c1f5-fd4e-4f0f-aa70-a8cc60a1ade5 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 14:59:42.298609 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-e809c1f5-fd4e-4f0f-aa70-a8cc60a1ade5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 14:59:42.329797 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80749968-3dd3-4036-8081-704125d47ae1 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:42.368267 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f94c1a0-311e-4490-95be-c915aabb78b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:42.369271 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f94c1a0-311e-4490-95be-c915aabb78b6 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:42.369778 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f94c1a0-311e-4490-95be-c915aabb78b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:42.370491 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f94c1a0-311e-4490-95be-c915aabb78b6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:42.371487 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f94c1a0-311e-4490-95be-c915aabb78b6 None admin] RBAC: Adding query filter params (interface=public, service_id=29bada023bc7467f915f45a4e65b9dc0, region_id=RegionOne) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:59:42.372087 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f94c1a0-311e-4490-95be-c915aabb78b6 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:42.372463 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 406/775] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Sep 23 14:59:42 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 23 14:59:42.374117 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f94c1a0-311e-4490-95be-c915aabb78b6 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:42.382642 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 370/776] 10.222.0.55 () {56 vars in 1251 bytes} [Fri Sep 23 14:59:42 2022] GET /identity/v3/endpoints?service_id=29bada023bc7467f915f45a4e65b9dc0&interface=public®ion_id=RegionOne => generated 194 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 14:59:42.388934 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e809c1f5-fd4e-4f0f-aa70-a8cc60a1ade5 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:42.487163 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:42.529907 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e809c1f5-fd4e-4f0f-aa70-a8cc60a1ade5 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:42.584680 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eef8fc3-a33b-40b6-bb82-965cb03f60c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:42.585297 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eef8fc3-a33b-40b6-bb82-965cb03f60c3 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:42.585573 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eef8fc3-a33b-40b6-bb82-965cb03f60c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:42.585845 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eef8fc3-a33b-40b6-bb82-965cb03f60c3 None admin] PATH_INFO: `/v3/services` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:42.586444 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eef8fc3-a33b-40b6-bb82-965cb03f60c3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:42.588501 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eef8fc3-a33b-40b6-bb82-965cb03f60c3 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:42.596576 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 407/777] 10.222.0.55 () {56 vars in 1087 bytes} [Fri Sep 23 14:59:42 2022] GET /identity/v3/services => generated 772 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:59:42.683518 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f94c1a0-311e-4490-95be-c915aabb78b6 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:42.697878 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eef8fc3-a33b-40b6-bb82-965cb03f60c3 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:42.808148 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f94c1a0-311e-4490-95be-c915aabb78b6 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:42.828435 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:42.850845 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6761af8-30c8-4900-85c6-665560384969 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:42.851885 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6761af8-30c8-4900-85c6-665560384969 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:42.852200 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6761af8-30c8-4900-85c6-665560384969 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:42.852476 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6761af8-30c8-4900-85c6-665560384969 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:42.866553 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6761af8-30c8-4900-85c6-665560384969 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 14:59:42.872354 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application [None req-b6761af8-30c8-4900-85c6-665560384969 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 23 14:59:42.872354 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 14:59:42.872354 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 14:59:42.872354 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 14:59:42.872354 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 14:59:42.872354 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 14:59:42.872354 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 14:59:42.872354 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 14:59:42.872354 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 14:59:42.872354 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 14:59:42.872354 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 14:59:42.872354 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 14:59:42.872354 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 14:59:42.872354 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 14:59:42.872354 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 14:59:42.872354 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 14:59:42.873313 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 14:59:42.873313 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 14:59:42.873313 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 14:59:42.873313 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 14:59:42.873313 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 14:59:42.873313 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 14:59:42.873313 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 14:59:42.873313 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 14:59:42.873313 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 14:59:42.873313 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 14:59:42.873313 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 14:59:42.873313 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application with Lock( Sep 23 14:59:42.873313 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 14:59:42.873313 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._enter() Sep 23 14:59:42.873313 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 14:59:42.873313 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 14:59:42.874159 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 14:59:42.874159 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.creator() Sep 23 14:59:42.874159 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 14:59:42.874159 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application created_value = creator( Sep 23 14:59:42.874159 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 14:59:42.874159 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 14:59:42.874159 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 14:59:42.874159 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 14:59:42.874159 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 14:59:42.874159 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 14:59:42.874159 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 23 14:59:42.874159 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Sep 23 14:59:42.874159 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 371/778] 10.222.0.55 () {56 vars in 1096 bytes} [Fri Sep 23 14:59:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 14:59:42.879387 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6761af8-30c8-4900-85c6-665560384969 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:42.891616 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eef8fc3-a33b-40b6-bb82-965cb03f60c3 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:42.937874 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2da518-c1b7-445f-bdbd-e384d1a505c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:42.938875 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2da518-c1b7-445f-bdbd-e384d1a505c6 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:42.939396 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2da518-c1b7-445f-bdbd-e384d1a505c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:42.939865 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2da518-c1b7-445f-bdbd-e384d1a505c6 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:42.940686 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2da518-c1b7-445f-bdbd-e384d1a505c6 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:42.942751 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2da518-c1b7-445f-bdbd-e384d1a505c6 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:42.945241 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:42.954862 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-7d2da518-c1b7-445f-bdbd-e384d1a505c6 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 23 14:59:42.954862 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 14:59:42.954862 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 23 14:59:42.954862 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 23 14:59:42.954862 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 23 14:59:42.954862 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 23 14:59:42.954862 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 23 14:59:42.954862 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 23 14:59:42.954862 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 23 14:59:42.954862 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Sep 23 14:59:42.954862 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 23 14:59:42.954862 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Sep 23 14:59:42.954862 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 14:59:42.954862 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 14:59:42.954862 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 14:59:42.954862 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 14:59:42.954862 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 14:59:42.961701 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 14:59:42.961701 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 14:59:42.961701 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 14:59:42.961701 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 14:59:42.961701 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 14:59:42.961701 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 14:59:42.961701 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 23 14:59:42.961701 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 23 14:59:42.961701 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 23 14:59:42.961701 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 23 14:59:42.961701 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 14:59:42.961701 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 14:59:42.961701 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 14:59:42.961701 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 14:59:42.961701 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 14:59:42.961701 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 14:59:42.963421 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 14:59:42.963421 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Sep 23 14:59:42.963421 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 14:59:42.963421 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Sep 23 14:59:42.963421 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 14:59:42.963421 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 14:59:42.963421 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 14:59:42.963421 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Sep 23 14:59:42.963421 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 14:59:42.963421 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Sep 23 14:59:42.963421 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 23 14:59:42.963421 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 23 14:59:42.963421 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 23 14:59:42.963421 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Sep 23 14:59:42.963421 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 408/779] 10.222.0.55 () {56 vars in 1129 bytes} [Fri Sep 23 14:59:42 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 14:59:42.967114 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2da518-c1b7-445f-bdbd-e384d1a505c6 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:42.993591 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6761af8-30c8-4900-85c6-665560384969 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:43.028776 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca1c4dc-67fd-4d05-8b44-d6ab0337d930 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:43.029383 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca1c4dc-67fd-4d05-8b44-d6ab0337d930 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:43.029663 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca1c4dc-67fd-4d05-8b44-d6ab0337d930 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:43.029991 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca1c4dc-67fd-4d05-8b44-d6ab0337d930 None admin] PATH_INFO: `/v3/roles` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:43.030649 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca1c4dc-67fd-4d05-8b44-d6ab0337d930 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:59:43.030946 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca1c4dc-67fd-4d05-8b44-d6ab0337d930 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:43.032339 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca1c4dc-67fd-4d05-8b44-d6ab0337d930 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:43.042627 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 372/780] 10.222.0.55 () {56 vars in 1099 bytes} [Fri Sep 23 14:59:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 14:59:43.047853 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca1c4dc-67fd-4d05-8b44-d6ab0337d930 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:43.096008 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:43.141159 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2da518-c1b7-445f-bdbd-e384d1a505c6 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:43.150694 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca1c4dc-67fd-4d05-8b44-d6ab0337d930 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:43.185142 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b601c7a2-9cb4-4d44-9199-762d25aeac1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:43.185776 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b601c7a2-9cb4-4d44-9199-762d25aeac1f None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:43.186081 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b601c7a2-9cb4-4d44-9199-762d25aeac1f None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:43.186356 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b601c7a2-9cb4-4d44-9199-762d25aeac1f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:43.197928 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b601c7a2-9cb4-4d44-9199-762d25aeac1f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:43.199377 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b601c7a2-9cb4-4d44-9199-762d25aeac1f None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:43.215711 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application [None req-b601c7a2-9cb4-4d44-9199-762d25aeac1f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 14:59:43.215711 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 14:59:43.215711 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 14:59:43.215711 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 14:59:43.215711 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 14:59:43.215711 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 14:59:43.215711 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 14:59:43.215711 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 14:59:43.215711 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 14:59:43.215711 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 14:59:43.215711 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 14:59:43.215711 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 14:59:43.215711 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 14:59:43.215711 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 14:59:43.215711 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 14:59:43.215711 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 14:59:43.216781 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 14:59:43.216781 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 14:59:43.216781 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 14:59:43.216781 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 14:59:43.216781 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 14:59:43.216781 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 14:59:43.216781 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 14:59:43.216781 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application with Lock( Sep 23 14:59:43.216781 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 14:59:43.216781 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._enter() Sep 23 14:59:43.216781 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 14:59:43.216781 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 14:59:43.216781 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 14:59:43.216781 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.creator() Sep 23 14:59:43.216781 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 14:59:43.216781 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application created_value = creator( Sep 23 14:59:43.216781 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 14:59:43.217702 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 14:59:43.217702 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 14:59:43.217702 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Sep 23 14:59:43.217702 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 373/781] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Sep 23 14:59:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 14:59:43.223388 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b601c7a2-9cb4-4d44-9199-762d25aeac1f None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:43.254560 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9229bdc8-b515-425b-9f24-0aa6fa46e46f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:43.255245 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9229bdc8-b515-425b-9f24-0aa6fa46e46f None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:43.255566 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9229bdc8-b515-425b-9f24-0aa6fa46e46f None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:43.255844 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9229bdc8-b515-425b-9f24-0aa6fa46e46f None admin] PATH_INFO: `/v3/services` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:43.256517 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9229bdc8-b515-425b-9f24-0aa6fa46e46f None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:59:43.256883 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9229bdc8-b515-425b-9f24-0aa6fa46e46f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:43.258304 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9229bdc8-b515-425b-9f24-0aa6fa46e46f None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:43.273960 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 409/782] 10.222.0.55 () {56 vars in 1124 bytes} [Fri Sep 23 14:59:42 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:59:43.280010 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9229bdc8-b515-425b-9f24-0aa6fa46e46f None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:43.320443 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:43.344158 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b601c7a2-9cb4-4d44-9199-762d25aeac1f None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:43.388776 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd6eb31-137a-4eb9-9a69-ff26a9c55638 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:43.389414 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd6eb31-137a-4eb9-9a69-ff26a9c55638 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:43.389693 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd6eb31-137a-4eb9-9a69-ff26a9c55638 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:43.389968 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd6eb31-137a-4eb9-9a69-ff26a9c55638 None admin] PATH_INFO: `/v3/domains` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:43.390619 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd6eb31-137a-4eb9-9a69-ff26a9c55638 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:59:43.390921 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd6eb31-137a-4eb9-9a69-ff26a9c55638 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:43.392326 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd6eb31-137a-4eb9-9a69-ff26a9c55638 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:43.405925 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 374/783] 10.222.0.55 () {56 vars in 1109 bytes} [Fri Sep 23 14:59:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 14:59:43.411647 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd6eb31-137a-4eb9-9a69-ff26a9c55638 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:43.432173 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:43.518030 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd6eb31-137a-4eb9-9a69-ff26a9c55638 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:43.520039 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9229bdc8-b515-425b-9f24-0aa6fa46e46f None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:43.554146 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f686d691-18e9-4288-b975-7b6d7d25e18a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:43.554844 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f686d691-18e9-4288-b975-7b6d7d25e18a None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:43.555124 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f686d691-18e9-4288-b975-7b6d7d25e18a None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:43.555395 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f686d691-18e9-4288-b975-7b6d7d25e18a None admin] PATH_INFO: `/v3/users/swift` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:43.569319 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f686d691-18e9-4288-b975-7b6d7d25e18a None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:43.572016 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f686d691-18e9-4288-b975-7b6d7d25e18a None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:43.585327 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2d74d6-10f1-4654-a0cd-117b8c713d4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:43.585954 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2d74d6-10f1-4654-a0cd-117b8c713d4d None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:43.586236 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2d74d6-10f1-4654-a0cd-117b8c713d4d None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:43.586509 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2d74d6-10f1-4654-a0cd-117b8c713d4d None admin] PATH_INFO: `/v3/services` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:43.588135 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2d74d6-10f1-4654-a0cd-117b8c713d4d None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:59:43.588454 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2d74d6-10f1-4654-a0cd-117b8c713d4d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:43.592291 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2d74d6-10f1-4654-a0cd-117b8c713d4d None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:43.597388 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application [None req-f686d691-18e9-4288-b975-7b6d7d25e18a None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 23 14:59:43.597388 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 14:59:43.597388 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 14:59:43.597388 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 14:59:43.597388 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 14:59:43.597388 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 14:59:43.597388 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 14:59:43.597388 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 14:59:43.597388 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 14:59:43.597388 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 14:59:43.597388 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 14:59:43.597388 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 14:59:43.597388 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 23 14:59:43.597388 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 23 14:59:43.597388 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 23 14:59:43.597388 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 23 14:59:43.598324 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 14:59:43.598324 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 14:59:43.598324 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 23 14:59:43.598324 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 14:59:43.598324 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 23 14:59:43.598324 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 14:59:43.598324 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 14:59:43.598324 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 14:59:43.598324 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 14:59:43.598324 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 14:59:43.598324 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 14:59:43.598324 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application with Lock( Sep 23 14:59:43.598324 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 14:59:43.598324 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._enter() Sep 23 14:59:43.598324 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 14:59:43.598324 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 14:59:43.598324 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 14:59:43.598324 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.creator() Sep 23 14:59:43.599267 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 14:59:43.599267 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application created_value = creator( Sep 23 14:59:43.599267 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 23 14:59:43.599267 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 23 14:59:43.599267 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 23 14:59:43.599267 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 23 14:59:43.599267 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 23 14:59:43.599267 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 23 14:59:43.599267 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 23 14:59:43.599267 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Sep 23 14:59:43.605887 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 375/784] 10.222.0.55 () {56 vars in 1096 bytes} [Fri Sep 23 14:59:43 2022] GET /identity/v3/users/swift => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 23 14:59:43.606155 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 410/785] 10.222.0.55 () {56 vars in 1124 bytes} [Fri Sep 23 14:59:43 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 14:59:43.608216 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f686d691-18e9-4288-b975-7b6d7d25e18a None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:43.611967 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2d74d6-10f1-4654-a0cd-117b8c713d4d None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:43.644568 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:43.725350 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f686d691-18e9-4288-b975-7b6d7d25e18a None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:43.762071 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4926536-4a26-427d-8a21-5260e9cffdcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:43.762877 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4926536-4a26-427d-8a21-5260e9cffdcd None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:43.763253 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4926536-4a26-427d-8a21-5260e9cffdcd None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:43.763625 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4926536-4a26-427d-8a21-5260e9cffdcd None admin] PATH_INFO: `/v3/users` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:43.764509 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4926536-4a26-427d-8a21-5260e9cffdcd None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:59:43.764917 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4926536-4a26-427d-8a21-5260e9cffdcd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:43.766547 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4926536-4a26-427d-8a21-5260e9cffdcd None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:43.786581 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 376/786] 10.222.0.55 () {56 vars in 1135 bytes} [Fri Sep 23 14:59:43 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:59:43.792970 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4926536-4a26-427d-8a21-5260e9cffdcd None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:43.847109 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2d74d6-10f1-4654-a0cd-117b8c713d4d None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:43.872891 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:43.878977 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:43.893123 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44589699-ff51-41a0-a153-9803dc4f3fcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:43.893766 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44589699-ff51-41a0-a153-9803dc4f3fcd None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:43.894021 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44589699-ff51-41a0-a153-9803dc4f3fcd None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:43.894300 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44589699-ff51-41a0-a153-9803dc4f3fcd None admin] PATH_INFO: `/v3/endpoints` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:43.895058 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44589699-ff51-41a0-a153-9803dc4f3fcd None admin] RBAC: Adding query filter params (interface=public, service_id=5dba32e33c7c4cfe8d212b0d7018a609, region_id=RegionOne) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:59:43.895316 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44589699-ff51-41a0-a153-9803dc4f3fcd None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:43.896732 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44589699-ff51-41a0-a153-9803dc4f3fcd None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:43.906745 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 411/787] 10.222.0.55 () {56 vars in 1251 bytes} [Fri Sep 23 14:59:43 2022] GET /identity/v3/endpoints?service_id=5dba32e33c7c4cfe8d212b0d7018a609&interface=public®ion_id=RegionOne => generated 194 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:59:43.908476 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521bab39-c29f-47de-aa32-01c86fef3841 None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:43.908765 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521bab39-c29f-47de-aa32-01c86fef3841 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:43.909053 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521bab39-c29f-47de-aa32-01c86fef3841 None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:43.912042 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 412/788] 10.222.0.55 () {54 vars in 927 bytes} [Fri Sep 23 14:59:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 14:59:43.913005 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521bab39-c29f-47de-aa32-01c86fef3841 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:43.926611 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4926536-4a26-427d-8a21-5260e9cffdcd None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:43.974638 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c91ef3-c487-4603-8d83-c4703c44b4c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:43.976107 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c91ef3-c487-4603-8d83-c4703c44b4c4 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:43.976518 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c91ef3-c487-4603-8d83-c4703c44b4c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:43.976870 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c91ef3-c487-4603-8d83-c4703c44b4c4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:43.992885 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c91ef3-c487-4603-8d83-c4703c44b4c4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:43.995610 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c91ef3-c487-4603-8d83-c4703c44b4c4 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:44.019748 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application [None req-75c91ef3-c487-4603-8d83-c4703c44b4c4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 14:59:44.019748 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 14:59:44.019748 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 14:59:44.019748 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 14:59:44.019748 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 14:59:44.019748 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 14:59:44.019748 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 14:59:44.019748 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 14:59:44.019748 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 14:59:44.019748 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 14:59:44.019748 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 14:59:44.019748 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 14:59:44.019748 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 14:59:44.019748 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 14:59:44.019748 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 14:59:44.019748 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 14:59:44.020879 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 14:59:44.020879 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 14:59:44.020879 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 14:59:44.020879 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 14:59:44.020879 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 14:59:44.020879 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 14:59:44.020879 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 14:59:44.020879 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application with Lock( Sep 23 14:59:44.020879 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 14:59:44.020879 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._enter() Sep 23 14:59:44.020879 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 14:59:44.020879 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 14:59:44.020879 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 14:59:44.020879 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.creator() Sep 23 14:59:44.020879 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 14:59:44.020879 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application created_value = creator( Sep 23 14:59:44.020879 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 14:59:44.021800 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 14:59:44.021800 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 14:59:44.021800 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Sep 23 14:59:44.026082 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 377/789] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Sep 23 14:59:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 23 14:59:44.027650 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2980980-db02-4640-894c-993ea8fc9db9 None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:44.027942 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2980980-db02-4640-894c-993ea8fc9db9 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:44.028246 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2980980-db02-4640-894c-993ea8fc9db9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:44.040545 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521bab39-c29f-47de-aa32-01c86fef3841 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:44.052232 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-a2980980-db02-4640-894c-993ea8fc9db9 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 14:59:44.053766 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-a2980980-db02-4640-894c-993ea8fc9db9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 14:59:44.090996 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0dc5388-046b-4dcc-89f9-e4ac8df06891 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:44.091619 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dc5388-046b-4dcc-89f9-e4ac8df06891 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:44.091907 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dc5388-046b-4dcc-89f9-e4ac8df06891 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:44.092268 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dc5388-046b-4dcc-89f9-e4ac8df06891 None admin] PATH_INFO: `/v3/services` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:44.092883 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0dc5388-046b-4dcc-89f9-e4ac8df06891 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:44.094323 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0dc5388-046b-4dcc-89f9-e4ac8df06891 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:44.100360 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 413/790] 10.222.0.55 () {56 vars in 1087 bytes} [Fri Sep 23 14:59:43 2022] GET /identity/v3/services => generated 772 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:59:44.101231 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0dc5388-046b-4dcc-89f9-e4ac8df06891 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:44.137307 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 378/791] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Sep 23 14:59:44 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 23 14:59:44.139092 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd4f8a3-71ef-4622-a500-fe0863ea377a None None] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:44.139644 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd4f8a3-71ef-4622-a500-fe0863ea377a None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:44.140141 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd4f8a3-71ef-4622-a500-fe0863ea377a None None] PATH_INFO: `/` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:44.142136 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 379/792] 10.222.0.55 () {54 vars in 927 bytes} [Fri Sep 23 14:59:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 14:59:44.152000 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edc1a6b-41ab-45cc-875e-43d91f363ebb None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:44.152000 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edc1a6b-41ab-45cc-875e-43d91f363ebb None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:44.152000 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edc1a6b-41ab-45cc-875e-43d91f363ebb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:44.185261 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-1edc1a6b-41ab-45cc-875e-43d91f363ebb None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 14:59:44.186479 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-1edc1a6b-41ab-45cc-875e-43d91f363ebb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 14:59:44.203205 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:44.258606 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0dc5388-046b-4dcc-89f9-e4ac8df06891 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:44.298818 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c83ff5-d5a5-4cae-b556-caae34769bef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:44.299442 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c83ff5-d5a5-4cae-b556-caae34769bef None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:44.299727 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c83ff5-d5a5-4cae-b556-caae34769bef None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:44.300025 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c83ff5-d5a5-4cae-b556-caae34769bef None admin] PATH_INFO: `/v3/domains` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:44.300747 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c83ff5-d5a5-4cae-b556-caae34769bef None admin] RBAC: Adding query filter params (name=Default) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:59:44.301053 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c83ff5-d5a5-4cae-b556-caae34769bef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:44.302746 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c83ff5-d5a5-4cae-b556-caae34769bef None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:44.314795 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:44.324063 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 414/793] 10.222.0.55 () {56 vars in 1109 bytes} [Fri Sep 23 14:59:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:59:44.330428 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c83ff5-d5a5-4cae-b556-caae34769bef None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:44.349817 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 380/794] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Sep 23 14:59:44 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 23 14:59:44.441128 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c83ff5-d5a5-4cae-b556-caae34769bef None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:44.456752 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:44.476114 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edc1a6b-41ab-45cc-875e-43d91f363ebb None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:44.478379 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b664dacd-f5e0-489f-a78f-b706f9830aba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:44.479034 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b664dacd-f5e0-489f-a78f-b706f9830aba None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:44.479312 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b664dacd-f5e0-489f-a78f-b706f9830aba None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:44.479585 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b664dacd-f5e0-489f-a78f-b706f9830aba None admin] PATH_INFO: `/v3/projects/service` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:44.487220 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b664dacd-f5e0-489f-a78f-b706f9830aba None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:44.489667 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b664dacd-f5e0-489f-a78f-b706f9830aba None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:44.503044 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-b664dacd-f5e0-489f-a78f-b706f9830aba None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 14:59:44.503044 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 14:59:44.503044 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 14:59:44.503044 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 14:59:44.503044 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 14:59:44.503044 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 14:59:44.503044 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 14:59:44.503044 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 14:59:44.503044 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 14:59:44.503044 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 14:59:44.503044 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 14:59:44.503044 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 14:59:44.503044 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 23 14:59:44.503044 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 23 14:59:44.503044 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 23 14:59:44.503970 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 23 14:59:44.503970 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 14:59:44.503970 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 14:59:44.503970 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 14:59:44.503970 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 14:59:44.503970 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 14:59:44.503970 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 14:59:44.503970 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 14:59:44.503970 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Sep 23 14:59:44.503970 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 14:59:44.503970 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Sep 23 14:59:44.503970 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 14:59:44.503970 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 14:59:44.503970 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 14:59:44.503970 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Sep 23 14:59:44.503970 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 14:59:44.503970 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Sep 23 14:59:44.504909 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 23 14:59:44.504909 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 23 14:59:44.504909 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 23 14:59:44.504909 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 23 14:59:44.504909 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 23 14:59:44.504909 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 23 14:59:44.504909 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 14:59:44.504909 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Sep 23 14:59:44.507768 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 415/795] 10.222.0.55 () {56 vars in 1111 bytes} [Fri Sep 23 14:59:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 14:59:44.510318 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b664dacd-f5e0-489f-a78f-b706f9830aba None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:44.623440 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b664dacd-f5e0-489f-a78f-b706f9830aba None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:44.633972 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edc1a6b-41ab-45cc-875e-43d91f363ebb None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:44.661240 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecbc639-122d-4cde-b2bf-b353db5eff3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:44.661863 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecbc639-122d-4cde-b2bf-b353db5eff3f None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:44.662136 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecbc639-122d-4cde-b2bf-b353db5eff3f None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:44.662422 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecbc639-122d-4cde-b2bf-b353db5eff3f None admin] PATH_INFO: `/v3/projects` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:44.663488 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ecbc639-122d-4cde-b2bf-b353db5eff3f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:59:44.663664 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ecbc639-122d-4cde-b2bf-b353db5eff3f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:44.665856 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ecbc639-122d-4cde-b2bf-b353db5eff3f None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:44.679729 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4156cf-bc1f-47b0-917d-fe247aea0e6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:44.681328 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4156cf-bc1f-47b0-917d-fe247aea0e6e None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:44.681825 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4156cf-bc1f-47b0-917d-fe247aea0e6e None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:44.682417 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4156cf-bc1f-47b0-917d-fe247aea0e6e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:44.684063 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 416/796] 10.222.0.55 () {56 vars in 1148 bytes} [Fri Sep 23 14:59:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 14:59:44.688590 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecbc639-122d-4cde-b2bf-b353db5eff3f None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:44.702860 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4156cf-bc1f-47b0-917d-fe247aea0e6e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 14:59:44.708682 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application [None req-fa4156cf-bc1f-47b0-917d-fe247aea0e6e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 23 14:59:44.708682 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 14:59:44.708682 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 14:59:44.708682 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 14:59:44.708682 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 14:59:44.708682 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 14:59:44.708682 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 14:59:44.708682 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 14:59:44.708682 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 14:59:44.708682 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 14:59:44.708682 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 14:59:44.708682 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 14:59:44.708682 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 14:59:44.708682 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 14:59:44.708682 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 14:59:44.708682 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 14:59:44.709634 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 14:59:44.709634 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 14:59:44.709634 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 14:59:44.709634 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 14:59:44.709634 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 14:59:44.709634 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 14:59:44.709634 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 14:59:44.709634 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 14:59:44.709634 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 14:59:44.709634 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 14:59:44.709634 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 14:59:44.709634 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application with Lock( Sep 23 14:59:44.709634 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 14:59:44.709634 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._enter() Sep 23 14:59:44.709634 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 14:59:44.709634 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 14:59:44.710486 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 14:59:44.710486 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.creator() Sep 23 14:59:44.710486 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 14:59:44.710486 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application created_value = creator( Sep 23 14:59:44.710486 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 14:59:44.710486 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 14:59:44.710486 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 14:59:44.710486 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 14:59:44.710486 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 14:59:44.710486 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 14:59:44.710486 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 23 14:59:44.710486 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Sep 23 14:59:44.710486 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 381/797] 10.222.0.55 () {56 vars in 1096 bytes} [Fri Sep 23 14:59:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 14:59:44.714943 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4156cf-bc1f-47b0-917d-fe247aea0e6e None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:44.739290 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:44.762707 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:44.812642 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecbc639-122d-4cde-b2bf-b353db5eff3f None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:44.830321 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4156cf-bc1f-47b0-917d-fe247aea0e6e None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:44.850706 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98945376-02ee-45a0-a0e0-4e3ed77fff4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:44.851767 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98945376-02ee-45a0-a0e0-4e3ed77fff4d None admin] REQUEST_METHOD: `PUT` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:44.852303 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98945376-02ee-45a0-a0e0-4e3ed77fff4d None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:44.853327 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98945376-02ee-45a0-a0e0-4e3ed77fff4d None admin] PATH_INFO: `/v3/projects/87b3e9d4b6474bd4a04ba59ef52bf3e2/users/2228c0b49a0249bbaf524a8c4e9e108b/roles/8e2919e79df040e8926f21607818b58c` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:44.855614 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98945376-02ee-45a0-a0e0-4e3ed77fff4d None admin] RBAC: Authorizing `identity:create_grant(project_id=87b3e9d4b6474bd4a04ba59ef52bf3e2, user_id=2228c0b49a0249bbaf524a8c4e9e108b, role_id=8e2919e79df040e8926f21607818b58c)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:44.857302 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98945376-02ee-45a0-a0e0-4e3ed77fff4d None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:44.868794 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a918021-803a-4a1f-a741-74fa05aa9b4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:44.869532 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a918021-803a-4a1f-a741-74fa05aa9b4f None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:44.869811 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a918021-803a-4a1f-a741-74fa05aa9b4f None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:44.876111 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a918021-803a-4a1f-a741-74fa05aa9b4f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:44.881403 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a918021-803a-4a1f-a741-74fa05aa9b4f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 14:59:44.886730 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application [None req-9a918021-803a-4a1f-a741-74fa05aa9b4f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 23 14:59:44.886730 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 14:59:44.886730 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 14:59:44.886730 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 14:59:44.886730 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 14:59:44.886730 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 14:59:44.886730 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 14:59:44.886730 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 14:59:44.886730 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 14:59:44.886730 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 14:59:44.886730 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 14:59:44.886730 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 14:59:44.886730 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 14:59:44.886730 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 14:59:44.886730 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 14:59:44.886730 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 14:59:44.887896 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 14:59:44.887896 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 14:59:44.887896 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 14:59:44.887896 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 14:59:44.887896 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 14:59:44.887896 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 14:59:44.887896 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 14:59:44.887896 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 14:59:44.887896 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 14:59:44.887896 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 14:59:44.887896 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 14:59:44.887896 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application with Lock( Sep 23 14:59:44.887896 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 14:59:44.887896 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._enter() Sep 23 14:59:44.887896 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 14:59:44.887896 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 14:59:44.889277 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 14:59:44.889277 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.creator() Sep 23 14:59:44.889277 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 14:59:44.889277 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application created_value = creator( Sep 23 14:59:44.889277 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 14:59:44.889277 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 14:59:44.889277 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 14:59:44.889277 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 14:59:44.889277 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 14:59:44.889277 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 14:59:44.889277 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 23 14:59:44.889277 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Sep 23 14:59:44.889277 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 382/798] 10.222.0.55 () {56 vars in 1102 bytes} [Fri Sep 23 14:59:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 14:59:44.889277 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a918021-803a-4a1f-a741-74fa05aa9b4f None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:44.936838 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 417/799] 10.222.0.55 () {58 vars in 1439 bytes} [Fri Sep 23 14:59:44 2022] PUT /identity/v3/projects/87b3e9d4b6474bd4a04ba59ef52bf3e2/users/2228c0b49a0249bbaf524a8c4e9e108b/roles/8e2919e79df040e8926f21607818b58c => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 14:59:44.938080 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98945376-02ee-45a0-a0e0-4e3ed77fff4d None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:45.079335 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:45.088743 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98945376-02ee-45a0-a0e0-4e3ed77fff4d None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:45.126167 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:45.131612 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59daacf8-4b9b-4fb2-a3c3-f39b1f327421 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:45.132323 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59daacf8-4b9b-4fb2-a3c3-f39b1f327421 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:45.132604 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59daacf8-4b9b-4fb2-a3c3-f39b1f327421 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:45.132878 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59daacf8-4b9b-4fb2-a3c3-f39b1f327421 None admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:45.133562 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59daacf8-4b9b-4fb2-a3c3-f39b1f327421 None admin] RBAC: Adding query filter params (name=service) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:59:45.134477 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59daacf8-4b9b-4fb2-a3c3-f39b1f327421 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:45.136788 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59daacf8-4b9b-4fb2-a3c3-f39b1f327421 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:45.142131 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a918021-803a-4a1f-a741-74fa05aa9b4f None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:45.147121 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 418/800] 10.222.0.55 () {56 vars in 1103 bytes} [Fri Sep 23 14:59:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:59:45.153166 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59daacf8-4b9b-4fb2-a3c3-f39b1f327421 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:45.199938 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7690c1c-0cf0-48b9-b8e5-3f5bdb39b7df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:45.201025 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7690c1c-0cf0-48b9-b8e5-3f5bdb39b7df None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:45.201522 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7690c1c-0cf0-48b9-b8e5-3f5bdb39b7df None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:45.202006 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7690c1c-0cf0-48b9-b8e5-3f5bdb39b7df None admin] PATH_INFO: `/v3/roles` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:45.202947 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7690c1c-0cf0-48b9-b8e5-3f5bdb39b7df None admin] RBAC: Adding query filter params (name=admin) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:59:45.203483 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7690c1c-0cf0-48b9-b8e5-3f5bdb39b7df None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:45.205626 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7690c1c-0cf0-48b9-b8e5-3f5bdb39b7df None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:45.217319 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 383/801] 10.222.0.55 () {56 vars in 1099 bytes} [Fri Sep 23 14:59:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:59:45.223191 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7690c1c-0cf0-48b9-b8e5-3f5bdb39b7df None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:45.289113 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59daacf8-4b9b-4fb2-a3c3-f39b1f327421 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:45.336190 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3de3c3-540f-467e-b2e1-ee99e0d99491 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:45.336761 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3de3c3-540f-467e-b2e1-ee99e0d99491 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:45.337602 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3de3c3-540f-467e-b2e1-ee99e0d99491 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:45.337879 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3de3c3-540f-467e-b2e1-ee99e0d99491 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:45.374396 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7690c1c-0cf0-48b9-b8e5-3f5bdb39b7df None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:45.415867 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6127a261-44e8-4fc1-9c84-bfced116acab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:45.416550 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6127a261-44e8-4fc1-9c84-bfced116acab None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:45.416876 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6127a261-44e8-4fc1-9c84-bfced116acab None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:45.417156 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6127a261-44e8-4fc1-9c84-bfced116acab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:45.431144 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6127a261-44e8-4fc1-9c84-bfced116acab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:45.432773 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6127a261-44e8-4fc1-9c84-bfced116acab None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:45.450936 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application [None req-6127a261-44e8-4fc1-9c84-bfced116acab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 14:59:45.450936 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 14:59:45.450936 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 14:59:45.450936 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 14:59:45.450936 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 14:59:45.450936 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 14:59:45.450936 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 14:59:45.450936 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 14:59:45.450936 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 14:59:45.450936 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 14:59:45.450936 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 14:59:45.450936 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 14:59:45.450936 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 14:59:45.450936 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 14:59:45.450936 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 14:59:45.450936 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 14:59:45.451955 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 14:59:45.451955 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 14:59:45.451955 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 14:59:45.451955 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 14:59:45.451955 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 14:59:45.451955 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 14:59:45.451955 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 14:59:45.451955 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application with Lock( Sep 23 14:59:45.451955 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 14:59:45.451955 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._enter() Sep 23 14:59:45.451955 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 14:59:45.451955 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 14:59:45.451955 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 14:59:45.451955 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.creator() Sep 23 14:59:45.451955 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 14:59:45.451955 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application created_value = creator( Sep 23 14:59:45.451955 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 14:59:45.452977 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 14:59:45.452977 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 14:59:45.452977 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Sep 23 14:59:45.452977 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 384/802] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Sep 23 14:59:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 23 14:59:45.453658 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea564242-8b4b-400b-9bfd-d99f84599314 None None] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:45.453939 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea564242-8b4b-400b-9bfd-d99f84599314 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:45.454215 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea564242-8b4b-400b-9bfd-d99f84599314 None None] PATH_INFO: `/` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:45.455788 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 385/803] 10.222.0.55 () {54 vars in 927 bytes} [Fri Sep 23 14:59:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 14:59:45.461649 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea564242-8b4b-400b-9bfd-d99f84599314 None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:45.475239 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3de3c3-540f-467e-b2e1-ee99e0d99491 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:45.476849 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3de3c3-540f-467e-b2e1-ee99e0d99491 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:45.488911 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:45.492057 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-fa3de3c3-540f-467e-b2e1-ee99e0d99491 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 14:59:45.492057 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 14:59:45.492057 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 14:59:45.492057 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 14:59:45.492057 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 14:59:45.492057 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 14:59:45.492057 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 14:59:45.492057 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 14:59:45.492057 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 14:59:45.492057 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 14:59:45.492057 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 14:59:45.492057 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 14:59:45.492057 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 14:59:45.492057 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 14:59:45.492057 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 14:59:45.492057 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 14:59:45.493838 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 14:59:45.493838 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 14:59:45.493838 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 14:59:45.493838 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 14:59:45.493838 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 14:59:45.493838 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 14:59:45.493838 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 14:59:45.493838 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Sep 23 14:59:45.493838 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 14:59:45.493838 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Sep 23 14:59:45.493838 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 14:59:45.493838 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 14:59:45.493838 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 14:59:45.493838 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Sep 23 14:59:45.493838 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 14:59:45.493838 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Sep 23 14:59:45.493838 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 14:59:45.495452 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 14:59:45.495452 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 14:59:45.495452 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Sep 23 14:59:45.499361 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 419/804] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Sep 23 14:59:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 342 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 14:59:45.500661 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d941545c-a2f0-4cae-9cc2-b86f2e6a70e1 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:45.500949 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d941545c-a2f0-4cae-9cc2-b86f2e6a70e1 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:45.501221 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d941545c-a2f0-4cae-9cc2-b86f2e6a70e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:45.526866 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-d941545c-a2f0-4cae-9cc2-b86f2e6a70e1 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 14:59:45.529008 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-d941545c-a2f0-4cae-9cc2-b86f2e6a70e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 14:59:45.531745 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:45.584866 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea564242-8b4b-400b-9bfd-d99f84599314 None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:45.623016 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 420/805] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Sep 23 14:59:45 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 23 14:59:45.623450 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45aaf6dd-431f-4aac-8a2d-7a84a135cb5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:45.624154 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d941545c-a2f0-4cae-9cc2-b86f2e6a70e1 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:45.624276 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45aaf6dd-431f-4aac-8a2d-7a84a135cb5a None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:45.624508 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45aaf6dd-431f-4aac-8a2d-7a84a135cb5a None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:45.624791 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45aaf6dd-431f-4aac-8a2d-7a84a135cb5a None admin] PATH_INFO: `/v3/domains` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:45.625482 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45aaf6dd-431f-4aac-8a2d-7a84a135cb5a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:59:45.625792 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45aaf6dd-431f-4aac-8a2d-7a84a135cb5a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:45.627581 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45aaf6dd-431f-4aac-8a2d-7a84a135cb5a None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:45.642576 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 386/806] 10.222.0.55 () {56 vars in 1109 bytes} [Fri Sep 23 14:59:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:59:45.647535 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45aaf6dd-431f-4aac-8a2d-7a84a135cb5a None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:45.756953 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d941545c-a2f0-4cae-9cc2-b86f2e6a70e1 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:45.794556 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17bb6e3-af47-4ab4-9bad-517e280fd40f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:45.795179 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17bb6e3-af47-4ab4-9bad-517e280fd40f None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:45.795461 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17bb6e3-af47-4ab4-9bad-517e280fd40f None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:45.795735 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17bb6e3-af47-4ab4-9bad-517e280fd40f None admin] PATH_INFO: `/v3/domains` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:45.796417 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17bb6e3-af47-4ab4-9bad-517e280fd40f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:59:45.796719 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17bb6e3-af47-4ab4-9bad-517e280fd40f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:45.798201 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17bb6e3-af47-4ab4-9bad-517e280fd40f None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:45.806149 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:45.816151 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 421/807] 10.222.0.55 () {56 vars in 1109 bytes} [Fri Sep 23 14:59:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:59:45.818110 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17bb6e3-af47-4ab4-9bad-517e280fd40f None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:45.831797 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:45.882368 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45aaf6dd-431f-4aac-8a2d-7a84a135cb5a None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:45.935814 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17bb6e3-af47-4ab4-9bad-517e280fd40f None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:45.959909 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20dc50c-fae2-4106-a867-5070d83bf544 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:45.960653 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20dc50c-fae2-4106-a867-5070d83bf544 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:45.960931 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20dc50c-fae2-4106-a867-5070d83bf544 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:45.961204 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20dc50c-fae2-4106-a867-5070d83bf544 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:45.976373 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20dc50c-fae2-4106-a867-5070d83bf544 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:45.977908 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20dc50c-fae2-4106-a867-5070d83bf544 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:45.992717 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3f1261-1f83-4e4a-af65-addd35f84263 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:45.993641 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3f1261-1f83-4e4a-af65-addd35f84263 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:45.993807 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3f1261-1f83-4e4a-af65-addd35f84263 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:45.994073 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3f1261-1f83-4e4a-af65-addd35f84263 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:46.002627 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3f1261-1f83-4e4a-af65-addd35f84263 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:46.004154 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3f1261-1f83-4e4a-af65-addd35f84263 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:46.018210 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-bf3f1261-1f83-4e4a-af65-addd35f84263 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 23 14:59:46.018210 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 14:59:46.018210 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 14:59:46.018210 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 14:59:46.018210 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 14:59:46.018210 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 14:59:46.018210 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 14:59:46.018210 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 14:59:46.018210 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 14:59:46.018210 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 14:59:46.018210 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 14:59:46.018210 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 14:59:46.018210 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 23 14:59:46.018210 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 23 14:59:46.018210 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 23 14:59:46.018210 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 23 14:59:46.019233 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 14:59:46.019233 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 14:59:46.019233 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 23 14:59:46.019233 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 14:59:46.019233 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 23 14:59:46.019233 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 14:59:46.019233 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 14:59:46.019233 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 14:59:46.019233 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 14:59:46.019233 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 14:59:46.019233 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 14:59:46.019233 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Sep 23 14:59:46.019233 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 14:59:46.019233 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Sep 23 14:59:46.019233 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 14:59:46.019233 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 14:59:46.019233 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 14:59:46.019233 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Sep 23 14:59:46.020225 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 14:59:46.020225 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Sep 23 14:59:46.020225 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 23 14:59:46.020225 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 23 14:59:46.020225 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 23 14:59:46.020225 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 23 14:59:46.020225 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 23 14:59:46.020225 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 23 14:59:46.020225 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 23 14:59:46.020225 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Sep 23 14:59:46.020225 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 422/808] 10.222.0.55 () {56 vars in 1117 bytes} [Fri Sep 23 14:59:45 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 14:59:46.021640 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3f1261-1f83-4e4a-af65-addd35f84263 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:46.092541 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:46.125339 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application [None req-b20dc50c-fae2-4106-a867-5070d83bf544 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 23 14:59:46.125339 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 14:59:46.125339 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 14:59:46.125339 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 14:59:46.125339 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 14:59:46.125339 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 14:59:46.125339 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 14:59:46.125339 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 14:59:46.125339 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 14:59:46.125339 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 14:59:46.125339 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 14:59:46.125339 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 14:59:46.125339 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 23 14:59:46.125339 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 23 14:59:46.125339 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 23 14:59:46.125339 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 23 14:59:46.126533 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 14:59:46.126533 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 14:59:46.126533 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 23 14:59:46.126533 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 14:59:46.126533 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 23 14:59:46.126533 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 14:59:46.126533 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 14:59:46.126533 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 14:59:46.126533 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 14:59:46.126533 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 14:59:46.126533 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 14:59:46.126533 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application with Lock( Sep 23 14:59:46.126533 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 14:59:46.126533 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._enter() Sep 23 14:59:46.126533 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 14:59:46.126533 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 14:59:46.126533 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 14:59:46.126533 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.creator() Sep 23 14:59:46.127706 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 14:59:46.127706 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application created_value = creator( Sep 23 14:59:46.127706 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 23 14:59:46.127706 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 23 14:59:46.127706 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 23 14:59:46.127706 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 23 14:59:46.127706 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 23 14:59:46.127706 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 23 14:59:46.127706 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 23 14:59:46.127706 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Sep 23 14:59:46.129976 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 387/809] 10.222.0.55 () {56 vars in 1093 bytes} [Fri Sep 23 14:59:45 2022] GET /identity/v3/users/nova => generated 82 bytes in 483 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 14:59:46.130976 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20dc50c-fae2-4106-a867-5070d83bf544 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:46.143860 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3f1261-1f83-4e4a-af65-addd35f84263 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:46.190554 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd99b2df-f1c7-4d25-a905-b885edfaec77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:46.191328 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd99b2df-f1c7-4d25-a905-b885edfaec77 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:46.191738 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd99b2df-f1c7-4d25-a905-b885edfaec77 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:46.192065 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd99b2df-f1c7-4d25-a905-b885edfaec77 None admin] PATH_INFO: `/v3/services/network` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:46.192659 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd99b2df-f1c7-4d25-a905-b885edfaec77 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:46.194759 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd99b2df-f1c7-4d25-a905-b885edfaec77 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:46.207462 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-cd99b2df-f1c7-4d25-a905-b885edfaec77 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 23 14:59:46.207462 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 14:59:46.207462 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 23 14:59:46.207462 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 23 14:59:46.207462 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 23 14:59:46.207462 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 23 14:59:46.207462 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 23 14:59:46.207462 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 23 14:59:46.207462 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 23 14:59:46.207462 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Sep 23 14:59:46.207462 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 23 14:59:46.207462 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Sep 23 14:59:46.207462 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 14:59:46.207462 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 14:59:46.207462 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 14:59:46.207462 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 14:59:46.207462 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 14:59:46.208639 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 14:59:46.208639 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 14:59:46.208639 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 14:59:46.208639 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 14:59:46.208639 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 14:59:46.208639 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 14:59:46.208639 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 23 14:59:46.208639 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 23 14:59:46.208639 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 23 14:59:46.208639 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 23 14:59:46.208639 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 14:59:46.208639 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 14:59:46.208639 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 14:59:46.208639 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 14:59:46.208639 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 14:59:46.208639 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 14:59:46.209555 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 14:59:46.209555 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Sep 23 14:59:46.209555 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 14:59:46.209555 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Sep 23 14:59:46.209555 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 14:59:46.209555 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 14:59:46.209555 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 14:59:46.209555 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Sep 23 14:59:46.209555 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 14:59:46.209555 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Sep 23 14:59:46.209555 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 23 14:59:46.209555 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 23 14:59:46.209555 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 23 14:59:46.209555 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Sep 23 14:59:46.214177 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 423/810] 10.222.0.55 () {56 vars in 1111 bytes} [Fri Sep 23 14:59:46 2022] GET /identity/v3/services/network => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 14:59:46.216247 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd99b2df-f1c7-4d25-a905-b885edfaec77 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:46.247790 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20dc50c-fae2-4106-a867-5070d83bf544 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:46.282809 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:46.288268 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8291bddf-4c5e-4ca1-bfb5-a66847118d7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:46.288887 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8291bddf-4c5e-4ca1-bfb5-a66847118d7c None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:46.289168 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8291bddf-4c5e-4ca1-bfb5-a66847118d7c None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:46.289443 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8291bddf-4c5e-4ca1-bfb5-a66847118d7c None admin] PATH_INFO: `/v3/users` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:46.290207 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8291bddf-4c5e-4ca1-bfb5-a66847118d7c None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:59:46.291067 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8291bddf-4c5e-4ca1-bfb5-a66847118d7c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:46.292947 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8291bddf-4c5e-4ca1-bfb5-a66847118d7c None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:46.320675 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 388/811] 10.222.0.55 () {56 vars in 1149 bytes} [Fri Sep 23 14:59:46 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:59:46.322697 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8291bddf-4c5e-4ca1-bfb5-a66847118d7c None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:46.358374 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd99b2df-f1c7-4d25-a905-b885edfaec77 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:46.393564 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:46.412191 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e862cbcc-1a31-489a-98e5-fa811b3bf5c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:46.414335 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e862cbcc-1a31-489a-98e5-fa811b3bf5c5 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:46.414844 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e862cbcc-1a31-489a-98e5-fa811b3bf5c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:46.415346 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e862cbcc-1a31-489a-98e5-fa811b3bf5c5 None admin] PATH_INFO: `/v3/users` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:46.416419 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e862cbcc-1a31-489a-98e5-fa811b3bf5c5 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:59:46.416950 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e862cbcc-1a31-489a-98e5-fa811b3bf5c5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:46.419235 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e862cbcc-1a31-489a-98e5-fa811b3bf5c5 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:46.446215 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 424/812] 10.222.0.55 () {56 vars in 1133 bytes} [Fri Sep 23 14:59:46 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:59:46.448489 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e862cbcc-1a31-489a-98e5-fa811b3bf5c5 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:46.452460 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8291bddf-4c5e-4ca1-bfb5-a66847118d7c None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:46.497629 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370863ff-a190-4b8d-a5cb-54964b3e24e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:46.498255 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370863ff-a190-4b8d-a5cb-54964b3e24e5 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:46.498538 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370863ff-a190-4b8d-a5cb-54964b3e24e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:46.498813 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370863ff-a190-4b8d-a5cb-54964b3e24e5 None admin] PATH_INFO: `/v3/services` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:46.499446 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370863ff-a190-4b8d-a5cb-54964b3e24e5 None admin] RBAC: Adding query filter params (name=network) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:59:46.499751 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370863ff-a190-4b8d-a5cb-54964b3e24e5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:46.501230 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370863ff-a190-4b8d-a5cb-54964b3e24e5 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:46.511664 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 389/813] 10.222.0.55 () {56 vars in 1112 bytes} [Fri Sep 23 14:59:46 2022] GET /identity/v3/services?name=network => generated 124 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:59:46.512640 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370863ff-a190-4b8d-a5cb-54964b3e24e5 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:46.570889 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:46.583751 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e862cbcc-1a31-489a-98e5-fa811b3bf5c5 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:46.629579 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab17dc9-10b1-4aca-9848-a1e354fcb9b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:46.630355 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab17dc9-10b1-4aca-9848-a1e354fcb9b2 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:46.630738 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab17dc9-10b1-4aca-9848-a1e354fcb9b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:46.631115 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab17dc9-10b1-4aca-9848-a1e354fcb9b2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:46.636794 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370863ff-a190-4b8d-a5cb-54964b3e24e5 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:46.644115 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab17dc9-10b1-4aca-9848-a1e354fcb9b2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:46.645695 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab17dc9-10b1-4aca-9848-a1e354fcb9b2 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:46.663070 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-aab17dc9-10b1-4aca-9848-a1e354fcb9b2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 14:59:46.663070 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 14:59:46.663070 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 14:59:46.663070 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 14:59:46.663070 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 14:59:46.663070 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 14:59:46.663070 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 14:59:46.663070 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 14:59:46.663070 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 14:59:46.663070 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 14:59:46.663070 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 14:59:46.663070 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 14:59:46.663070 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 14:59:46.663070 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 14:59:46.663070 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 14:59:46.663070 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 14:59:46.664276 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 14:59:46.664276 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 14:59:46.664276 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 14:59:46.664276 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 14:59:46.664276 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 14:59:46.664276 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 14:59:46.664276 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 14:59:46.664276 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Sep 23 14:59:46.664276 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 14:59:46.664276 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Sep 23 14:59:46.664276 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 14:59:46.664276 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 14:59:46.664276 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 14:59:46.664276 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Sep 23 14:59:46.664276 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 14:59:46.664276 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Sep 23 14:59:46.664276 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 14:59:46.666189 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 14:59:46.666189 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 14:59:46.666189 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Sep 23 14:59:46.671692 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 425/814] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Sep 23 14:59:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 14:59:46.672884 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab17dc9-10b1-4aca-9848-a1e354fcb9b2 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:46.681822 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9b5148-7116-4b99-b1d1-05752f38ae96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:46.682663 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9b5148-7116-4b99-b1d1-05752f38ae96 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:46.683179 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9b5148-7116-4b99-b1d1-05752f38ae96 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:46.683628 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9b5148-7116-4b99-b1d1-05752f38ae96 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:46.713279 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:46.758551 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9b5148-7116-4b99-b1d1-05752f38ae96 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:46.760320 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9b5148-7116-4b99-b1d1-05752f38ae96 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:46.787347 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab17dc9-10b1-4aca-9848-a1e354fcb9b2 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:46.817353 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:46.825441 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77b2dba-4c00-49c0-ae9d-807b73cee730 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:46.825845 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77b2dba-4c00-49c0-ae9d-807b73cee730 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:46.826192 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77b2dba-4c00-49c0-ae9d-807b73cee730 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:46.826502 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77b2dba-4c00-49c0-ae9d-807b73cee730 None admin] PATH_INFO: `/v3/services` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:46.827129 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77b2dba-4c00-49c0-ae9d-807b73cee730 None admin] RBAC: Adding query filter params (type=network) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:59:46.827435 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77b2dba-4c00-49c0-ae9d-807b73cee730 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:46.828803 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77b2dba-4c00-49c0-ae9d-807b73cee730 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:46.833758 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 426/815] 10.222.0.55 () {56 vars in 1112 bytes} [Fri Sep 23 14:59:46 2022] GET /identity/v3/services?type=network => generated 350 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:59:46.834534 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77b2dba-4c00-49c0-ae9d-807b73cee730 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:46.870922 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 390/816] 10.222.0.55 () {58 vars in 1282 bytes} [Fri Sep 23 14:59:46 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 23 14:59:46.872254 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e42963e-020a-4542-a007-1e9cc318afa9 None None] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:46.872539 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e42963e-020a-4542-a007-1e9cc318afa9 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:46.872812 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e42963e-020a-4542-a007-1e9cc318afa9 None None] PATH_INFO: `/` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:46.874175 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 391/817] 10.222.0.55 () {54 vars in 927 bytes} [Fri Sep 23 14:59:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 14:59:46.878754 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e42963e-020a-4542-a007-1e9cc318afa9 None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:46.952160 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:46.959387 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77b2dba-4c00-49c0-ae9d-807b73cee730 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:46.982777 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e42963e-020a-4542-a007-1e9cc318afa9 None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:47.002829 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adeedf0c-3619-48d4-968c-afb0c024f127 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:47.003523 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adeedf0c-3619-48d4-968c-afb0c024f127 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:47.003720 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adeedf0c-3619-48d4-968c-afb0c024f127 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:47.004320 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adeedf0c-3619-48d4-968c-afb0c024f127 None admin] PATH_INFO: `/v3/domains` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:47.004634 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adeedf0c-3619-48d4-968c-afb0c024f127 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:59:47.004932 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adeedf0c-3619-48d4-968c-afb0c024f127 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:47.006325 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adeedf0c-3619-48d4-968c-afb0c024f127 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:47.019566 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 427/818] 10.222.0.55 () {56 vars in 1109 bytes} [Fri Sep 23 14:59:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:59:47.020427 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adeedf0c-3619-48d4-968c-afb0c024f127 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:47.025987 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2cc87b-10d4-4485-944a-d03991c0df72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:47.026613 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2cc87b-10d4-4485-944a-d03991c0df72 None admin] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:47.026915 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2cc87b-10d4-4485-944a-d03991c0df72 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:47.027188 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2cc87b-10d4-4485-944a-d03991c0df72 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:47.027861 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2cc87b-10d4-4485-944a-d03991c0df72 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:47.029856 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2cc87b-10d4-4485-944a-d03991c0df72 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:47.131711 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adeedf0c-3619-48d4-968c-afb0c024f127 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:47.161044 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 392/819] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Sep 23 14:59:46 2022] POST /identity/v3/endpoints => generated 337 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 14:59:47.162437 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfe4657-1bfa-4b3a-a14e-b13037897e91 None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:47.162726 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfe4657-1bfa-4b3a-a14e-b13037897e91 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:47.163003 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfe4657-1bfa-4b3a-a14e-b13037897e91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:47.188900 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bdbebb-5036-4c06-85c3-f9d313564b28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:47.190117 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bdbebb-5036-4c06-85c3-f9d313564b28 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:47.191435 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bdbebb-5036-4c06-85c3-f9d313564b28 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:47.191972 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bdbebb-5036-4c06-85c3-f9d313564b28 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:47.197535 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-2dfe4657-1bfa-4b3a-a14e-b13037897e91 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 14:59:47.198775 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-2dfe4657-1bfa-4b3a-a14e-b13037897e91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 14:59:47.203172 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:47.209696 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6bdbebb-5036-4c06-85c3-f9d313564b28 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:47.212439 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6bdbebb-5036-4c06-85c3-f9d313564b28 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:47.236393 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-a6bdbebb-5036-4c06-85c3-f9d313564b28 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 14:59:47.236393 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 14:59:47.236393 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 14:59:47.236393 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 14:59:47.236393 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 14:59:47.236393 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 14:59:47.236393 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 14:59:47.236393 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 14:59:47.236393 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 14:59:47.236393 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 14:59:47.236393 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 14:59:47.236393 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 14:59:47.236393 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 14:59:47.236393 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 14:59:47.236393 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 14:59:47.236393 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 14:59:47.237499 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 14:59:47.237499 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 14:59:47.237499 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 14:59:47.237499 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 14:59:47.237499 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 14:59:47.237499 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 14:59:47.237499 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 14:59:47.237499 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Sep 23 14:59:47.237499 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 14:59:47.237499 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Sep 23 14:59:47.237499 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 14:59:47.237499 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 14:59:47.237499 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 14:59:47.237499 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Sep 23 14:59:47.237499 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 14:59:47.237499 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Sep 23 14:59:47.237499 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 14:59:47.238412 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 14:59:47.238412 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 14:59:47.238412 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Sep 23 14:59:47.241657 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 428/820] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Sep 23 14:59:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 14:59:47.242919 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bdbebb-5036-4c06-85c3-f9d313564b28 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:47.248652 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:47.288530 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 393/821] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Sep 23 14:59:47 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 23 14:59:47.289926 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfe4657-1bfa-4b3a-a14e-b13037897e91 None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:47.421974 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bdbebb-5036-4c06-85c3-f9d313564b28 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:47.433310 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfe4657-1bfa-4b3a-a14e-b13037897e91 None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:47.470226 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ce44e5-2e28-4135-bd22-918347eb014a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:47.470226 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ce44e5-2e28-4135-bd22-918347eb014a None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:47.470226 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ce44e5-2e28-4135-bd22-918347eb014a None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:47.470226 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ce44e5-2e28-4135-bd22-918347eb014a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:47.482658 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ce44e5-2e28-4135-bd22-918347eb014a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:47.482658 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ce44e5-2e28-4135-bd22-918347eb014a None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:47.489717 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c17442-01fb-47d8-bc1d-f3fac91e950d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:47.490348 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c17442-01fb-47d8-bc1d-f3fac91e950d None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:47.490627 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c17442-01fb-47d8-bc1d-f3fac91e950d None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:47.490901 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c17442-01fb-47d8-bc1d-f3fac91e950d None admin] PATH_INFO: `/v3/domains` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:47.491547 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c17442-01fb-47d8-bc1d-f3fac91e950d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:59:47.491849 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c17442-01fb-47d8-bc1d-f3fac91e950d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:47.493893 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c17442-01fb-47d8-bc1d-f3fac91e950d None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:47.495529 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-b7ce44e5-2e28-4135-bd22-918347eb014a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 14:59:47.495529 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 14:59:47.495529 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 14:59:47.495529 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 14:59:47.495529 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 14:59:47.495529 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 14:59:47.495529 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 14:59:47.495529 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 14:59:47.495529 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 14:59:47.495529 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 14:59:47.495529 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 14:59:47.495529 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 14:59:47.495529 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 23 14:59:47.495529 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 23 14:59:47.495529 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 23 14:59:47.496398 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 23 14:59:47.496398 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 14:59:47.496398 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 14:59:47.496398 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 14:59:47.496398 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 14:59:47.496398 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 14:59:47.496398 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 14:59:47.496398 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 14:59:47.496398 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Sep 23 14:59:47.496398 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 14:59:47.496398 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Sep 23 14:59:47.496398 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 14:59:47.496398 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 14:59:47.496398 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 14:59:47.496398 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Sep 23 14:59:47.496398 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 14:59:47.496398 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Sep 23 14:59:47.497288 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 23 14:59:47.497288 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 23 14:59:47.497288 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 23 14:59:47.497288 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 23 14:59:47.497288 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 23 14:59:47.497288 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 23 14:59:47.497288 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 14:59:47.497288 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Sep 23 14:59:47.497288 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 429/822] 10.222.0.55 () {56 vars in 1111 bytes} [Fri Sep 23 14:59:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 14:59:47.498655 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cc3824-ef10-42bf-bb30-b3e8fbd2d85e None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:47.498655 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cc3824-ef10-42bf-bb30-b3e8fbd2d85e None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:47.498655 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cc3824-ef10-42bf-bb30-b3e8fbd2d85e None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:47.501674 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 430/823] 10.222.0.55 () {54 vars in 927 bytes} [Fri Sep 23 14:59:47 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 14:59:47.505352 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07015e8d-4584-431b-ba20-5c637dbb1ec9 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:47.505632 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07015e8d-4584-431b-ba20-5c637dbb1ec9 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:47.506462 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07015e8d-4584-431b-ba20-5c637dbb1ec9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:47.508326 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:47.516139 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 394/824] 10.222.0.55 () {56 vars in 1109 bytes} [Fri Sep 23 14:59:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:59:47.517167 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c17442-01fb-47d8-bc1d-f3fac91e950d None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:47.530093 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-07015e8d-4584-431b-ba20-5c637dbb1ec9 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 14:59:47.532063 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-07015e8d-4584-431b-ba20-5c637dbb1ec9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 14:59:47.597316 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:47.605387 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 431/825] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Sep 23 14:59:47 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 23 14:59:47.606197 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07015e8d-4584-431b-ba20-5c637dbb1ec9 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:47.622392 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c17442-01fb-47d8-bc1d-f3fac91e950d None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:47.659058 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cef997d-42e1-4fde-b939-95592c7127cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:47.660027 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cef997d-42e1-4fde-b939-95592c7127cc None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:47.660297 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cef997d-42e1-4fde-b939-95592c7127cc None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:47.660571 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cef997d-42e1-4fde-b939-95592c7127cc None admin] PATH_INFO: `/v3/projects` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:47.661261 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cef997d-42e1-4fde-b939-95592c7127cc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:59:47.661565 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cef997d-42e1-4fde-b939-95592c7127cc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:47.662989 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cef997d-42e1-4fde-b939-95592c7127cc None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:47.676126 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 395/826] 10.222.0.55 () {56 vars in 1148 bytes} [Fri Sep 23 14:59:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:59:47.677190 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cef997d-42e1-4fde-b939-95592c7127cc None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:47.715573 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07015e8d-4584-431b-ba20-5c637dbb1ec9 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:47.738854 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:47.753353 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0153061-42c9-4412-b552-a5bb53ab7213 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:47.753989 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0153061-42c9-4412-b552-a5bb53ab7213 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:47.754270 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0153061-42c9-4412-b552-a5bb53ab7213 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:47.754543 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0153061-42c9-4412-b552-a5bb53ab7213 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:47.762239 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0153061-42c9-4412-b552-a5bb53ab7213 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:47.763680 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0153061-42c9-4412-b552-a5bb53ab7213 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:47.774954 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-f0153061-42c9-4412-b552-a5bb53ab7213 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 14:59:47.774954 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 14:59:47.774954 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 14:59:47.774954 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 14:59:47.774954 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 14:59:47.774954 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 14:59:47.774954 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 14:59:47.774954 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 14:59:47.774954 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 14:59:47.774954 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 14:59:47.774954 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 14:59:47.774954 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 14:59:47.774954 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 23 14:59:47.774954 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 23 14:59:47.774954 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 23 14:59:47.775908 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 23 14:59:47.775908 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 14:59:47.775908 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 14:59:47.775908 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 14:59:47.775908 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 14:59:47.775908 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 14:59:47.775908 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 14:59:47.775908 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 14:59:47.775908 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Sep 23 14:59:47.775908 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 14:59:47.775908 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Sep 23 14:59:47.775908 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 14:59:47.775908 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 14:59:47.775908 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 14:59:47.775908 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Sep 23 14:59:47.775908 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 14:59:47.775908 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Sep 23 14:59:47.781857 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 23 14:59:47.781857 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 23 14:59:47.781857 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 23 14:59:47.781857 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 23 14:59:47.781857 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 23 14:59:47.781857 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 23 14:59:47.781857 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 14:59:47.781857 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Sep 23 14:59:47.781857 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 432/827] 10.222.0.55 () {56 vars in 1111 bytes} [Fri Sep 23 14:59:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 14:59:47.781857 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0153061-42c9-4412-b552-a5bb53ab7213 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:47.786916 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cef997d-42e1-4fde-b939-95592c7127cc None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:47.823057 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328fb4aa-51ea-40f8-ae76-64d3cba343ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:47.824052 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328fb4aa-51ea-40f8-ae76-64d3cba343ce None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:47.824518 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328fb4aa-51ea-40f8-ae76-64d3cba343ce None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:47.824975 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328fb4aa-51ea-40f8-ae76-64d3cba343ce None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:47.825733 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328fb4aa-51ea-40f8-ae76-64d3cba343ce None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:47.827315 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328fb4aa-51ea-40f8-ae76-64d3cba343ce None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:47.836300 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application [None req-328fb4aa-51ea-40f8-ae76-64d3cba343ce None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 23 14:59:47.836300 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 14:59:47.836300 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 23 14:59:47.836300 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 23 14:59:47.836300 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 23 14:59:47.836300 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 23 14:59:47.836300 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 23 14:59:47.836300 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 23 14:59:47.836300 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 23 14:59:47.836300 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Sep 23 14:59:47.836300 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 23 14:59:47.836300 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Sep 23 14:59:47.836300 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 14:59:47.836300 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 14:59:47.836300 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 14:59:47.836300 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 14:59:47.836300 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 14:59:47.837358 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 14:59:47.837358 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 14:59:47.837358 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 14:59:47.837358 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 14:59:47.837358 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 14:59:47.837358 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 14:59:47.837358 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 23 14:59:47.837358 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 23 14:59:47.837358 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 23 14:59:47.837358 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 23 14:59:47.837358 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 14:59:47.837358 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 14:59:47.837358 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 14:59:47.837358 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 14:59:47.837358 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 14:59:47.837358 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 14:59:47.838196 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 14:59:47.838196 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application with Lock( Sep 23 14:59:47.838196 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 14:59:47.838196 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._enter() Sep 23 14:59:47.838196 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 14:59:47.838196 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 14:59:47.838196 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 14:59:47.838196 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.creator() Sep 23 14:59:47.838196 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 14:59:47.838196 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application created_value = creator( Sep 23 14:59:47.838196 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 23 14:59:47.838196 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 23 14:59:47.838196 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 23 14:59:47.838196 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Sep 23 14:59:47.838196 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 396/828] 10.222.0.55 () {56 vars in 1129 bytes} [Fri Sep 23 14:59:47 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 14:59:47.839009 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328fb4aa-51ea-40f8-ae76-64d3cba343ce None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:47.846820 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:47.894733 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0153061-42c9-4412-b552-a5bb53ab7213 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:47.929441 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2da060-5598-40cc-897b-8d02c3e1420d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:47.930125 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2da060-5598-40cc-897b-8d02c3e1420d None admin] REQUEST_METHOD: `PUT` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:47.930409 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2da060-5598-40cc-897b-8d02c3e1420d None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:47.931220 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2da060-5598-40cc-897b-8d02c3e1420d None admin] PATH_INFO: `/v3/projects/87b3e9d4b6474bd4a04ba59ef52bf3e2/users/68206a6b4f244d45bf7bf1504396e430/roles/0665869b71fa40b0b552efdd57faf023` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:47.942979 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328fb4aa-51ea-40f8-ae76-64d3cba343ce None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:47.947271 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2da060-5598-40cc-897b-8d02c3e1420d None admin] RBAC: Authorizing `identity:create_grant(project_id=87b3e9d4b6474bd4a04ba59ef52bf3e2, user_id=68206a6b4f244d45bf7bf1504396e430, role_id=0665869b71fa40b0b552efdd57faf023)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:47.948727 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2da060-5598-40cc-897b-8d02c3e1420d None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:47.966897 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:47.977123 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf04e84-f2ee-4cf8-b5b1-9ee6b88cd01e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:47.977764 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf04e84-f2ee-4cf8-b5b1-9ee6b88cd01e None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:47.978044 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf04e84-f2ee-4cf8-b5b1-9ee6b88cd01e None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:47.978315 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf04e84-f2ee-4cf8-b5b1-9ee6b88cd01e None admin] PATH_INFO: `/v3/projects` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:47.978998 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf04e84-f2ee-4cf8-b5b1-9ee6b88cd01e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:59:47.979300 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf04e84-f2ee-4cf8-b5b1-9ee6b88cd01e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:47.980737 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf04e84-f2ee-4cf8-b5b1-9ee6b88cd01e None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:47.992228 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 397/829] 10.222.0.55 () {56 vars in 1148 bytes} [Fri Sep 23 14:59:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:59:47.993036 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf04e84-f2ee-4cf8-b5b1-9ee6b88cd01e None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:48.089850 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf04e84-f2ee-4cf8-b5b1-9ee6b88cd01e None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:48.103395 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 433/830] 10.222.0.55 () {58 vars in 1439 bytes} [Fri Sep 23 14:59:47 2022] PUT /identity/v3/projects/87b3e9d4b6474bd4a04ba59ef52bf3e2/users/68206a6b4f244d45bf7bf1504396e430/roles/0665869b71fa40b0b552efdd57faf023 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 14:59:48.104354 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2da060-5598-40cc-897b-8d02c3e1420d None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:48.126359 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3f73f4-6279-4421-9da6-f63e6107bc24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:48.126991 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3f73f4-6279-4421-9da6-f63e6107bc24 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:48.127273 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3f73f4-6279-4421-9da6-f63e6107bc24 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:48.127549 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3f73f4-6279-4421-9da6-f63e6107bc24 None admin] PATH_INFO: `/v3/services` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:48.128236 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3f73f4-6279-4421-9da6-f63e6107bc24 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:59:48.128546 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3f73f4-6279-4421-9da6-f63e6107bc24 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:48.129964 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3f73f4-6279-4421-9da6-f63e6107bc24 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:48.140204 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 398/831] 10.222.0.55 () {56 vars in 1124 bytes} [Fri Sep 23 14:59:47 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:59:48.140204 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3f73f4-6279-4421-9da6-f63e6107bc24 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:48.193912 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:48.234937 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2da060-5598-40cc-897b-8d02c3e1420d None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:48.237763 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:48.248543 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3f73f4-6279-4421-9da6-f63e6107bc24 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:48.281475 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dbfe2c-b3df-4c42-a215-19fba0333c5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:48.281879 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dbfe2c-b3df-4c42-a215-19fba0333c5d None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:48.282014 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dbfe2c-b3df-4c42-a215-19fba0333c5d None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:48.282239 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dbfe2c-b3df-4c42-a215-19fba0333c5d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:48.295064 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f517057-c8ce-426d-b7e6-dd119a8893f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:48.295683 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f517057-c8ce-426d-b7e6-dd119a8893f9 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:48.295959 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f517057-c8ce-426d-b7e6-dd119a8893f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:48.296261 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f517057-c8ce-426d-b7e6-dd119a8893f9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:48.296503 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40dbfe2c-b3df-4c42-a215-19fba0333c5d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 14:59:48.296981 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f517057-c8ce-426d-b7e6-dd119a8893f9 None admin] RBAC: Adding query filter params (role.id=8e2919e79df040e8926f21607818b58c, scope.project.id=87b3e9d4b6474bd4a04ba59ef52bf3e2, user.id=b3c6c7a1325a47be924268e1aa1a7a18) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:59:48.297288 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f517057-c8ce-426d-b7e6-dd119a8893f9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:48.298699 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f517057-c8ce-426d-b7e6-dd119a8893f9 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:48.301392 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-40dbfe2c-b3df-4c42-a215-19fba0333c5d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 23 14:59:48.301392 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 14:59:48.301392 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 14:59:48.301392 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 14:59:48.301392 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 14:59:48.301392 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 14:59:48.301392 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 14:59:48.301392 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 14:59:48.301392 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 14:59:48.301392 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 14:59:48.301392 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 14:59:48.301392 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 14:59:48.301392 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 14:59:48.301392 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 14:59:48.301392 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 14:59:48.301392 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 14:59:48.302293 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 14:59:48.302293 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 14:59:48.302293 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 14:59:48.302293 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 14:59:48.302293 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 14:59:48.302293 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 14:59:48.302293 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 14:59:48.302293 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 14:59:48.302293 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 14:59:48.302293 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 14:59:48.302293 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 14:59:48.302293 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Sep 23 14:59:48.302293 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 14:59:48.302293 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Sep 23 14:59:48.302293 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 14:59:48.302293 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 14:59:48.303141 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 14:59:48.303141 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Sep 23 14:59:48.303141 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 14:59:48.303141 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Sep 23 14:59:48.303141 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 14:59:48.303141 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 14:59:48.303141 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 14:59:48.303141 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 14:59:48.303141 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 14:59:48.303141 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 14:59:48.303141 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 23 14:59:48.303141 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Sep 23 14:59:48.303141 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 434/832] 10.222.0.55 () {56 vars in 1096 bytes} [Fri Sep 23 14:59:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 14:59:48.303832 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dbfe2c-b3df-4c42-a215-19fba0333c5d None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:48.310538 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 399/833] 10.222.0.55 () {56 vars in 1374 bytes} [Fri Sep 23 14:59:48 2022] GET /identity/v3/role_assignments?user.id=b3c6c7a1325a47be924268e1aa1a7a18&scope.project.id=87b3e9d4b6474bd4a04ba59ef52bf3e2&role.id=8e2919e79df040e8926f21607818b58c => generated 608 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:59:48.311378 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f517057-c8ce-426d-b7e6-dd119a8893f9 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:48.439167 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dbfe2c-b3df-4c42-a215-19fba0333c5d None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:48.455514 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f517057-c8ce-426d-b7e6-dd119a8893f9 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:48.480697 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f2b107-974b-4084-989d-9060194a7161 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:48.481324 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f2b107-974b-4084-989d-9060194a7161 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:48.481612 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f2b107-974b-4084-989d-9060194a7161 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:48.481891 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f2b107-974b-4084-989d-9060194a7161 None admin] PATH_INFO: `/v3/services` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:48.482613 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f2b107-974b-4084-989d-9060194a7161 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:59:48.482922 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f2b107-974b-4084-989d-9060194a7161 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:48.484368 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f2b107-974b-4084-989d-9060194a7161 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:48.484915 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:48.490038 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 435/834] 10.222.0.55 () {56 vars in 1124 bytes} [Fri Sep 23 14:59:48 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:59:48.497807 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0feba308-becb-4912-8bac-c60da4953497 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:48.506240 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f2b107-974b-4084-989d-9060194a7161 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:48.506394 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0feba308-becb-4912-8bac-c60da4953497 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:48.506394 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0feba308-becb-4912-8bac-c60da4953497 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:48.506394 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0feba308-becb-4912-8bac-c60da4953497 None admin] PATH_INFO: `/v3/roles` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:48.506394 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0feba308-becb-4912-8bac-c60da4953497 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:59:48.506394 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0feba308-becb-4912-8bac-c60da4953497 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:48.506394 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0feba308-becb-4912-8bac-c60da4953497 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:48.514933 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 400/835] 10.222.0.55 () {56 vars in 1099 bytes} [Fri Sep 23 14:59:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:59:48.520751 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0feba308-becb-4912-8bac-c60da4953497 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:48.531559 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:48.625759 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f2b107-974b-4084-989d-9060194a7161 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:48.664804 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da1a5a0-f23a-48d2-9b72-3e06600ea334 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:48.666381 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da1a5a0-f23a-48d2-9b72-3e06600ea334 None admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:48.666942 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da1a5a0-f23a-48d2-9b72-3e06600ea334 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:48.667429 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da1a5a0-f23a-48d2-9b72-3e06600ea334 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:48.668511 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da1a5a0-f23a-48d2-9b72-3e06600ea334 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:48.670747 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da1a5a0-f23a-48d2-9b72-3e06600ea334 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:48.679108 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0feba308-becb-4912-8bac-c60da4953497 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:48.718084 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65688afb-3dd1-4c71-899c-e65b07616495 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:48.718726 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65688afb-3dd1-4c71-899c-e65b07616495 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:48.718999 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65688afb-3dd1-4c71-899c-e65b07616495 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:48.719271 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65688afb-3dd1-4c71-899c-e65b07616495 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:48.730837 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65688afb-3dd1-4c71-899c-e65b07616495 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:48.733778 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65688afb-3dd1-4c71-899c-e65b07616495 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:48.747500 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application [None req-65688afb-3dd1-4c71-899c-e65b07616495 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 14:59:48.747500 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 14:59:48.747500 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 14:59:48.747500 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 14:59:48.747500 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 14:59:48.747500 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 14:59:48.747500 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 14:59:48.747500 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 14:59:48.747500 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 14:59:48.747500 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 14:59:48.747500 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 14:59:48.747500 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 14:59:48.747500 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 14:59:48.747500 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 14:59:48.747500 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 14:59:48.747500 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 14:59:48.748807 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 14:59:48.748807 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 14:59:48.748807 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 14:59:48.748807 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 14:59:48.748807 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 14:59:48.748807 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 14:59:48.748807 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 14:59:48.748807 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application with Lock( Sep 23 14:59:48.748807 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 14:59:48.748807 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._enter() Sep 23 14:59:48.748807 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 14:59:48.748807 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 14:59:48.748807 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 14:59:48.748807 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.creator() Sep 23 14:59:48.748807 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 14:59:48.748807 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application created_value = creator( Sep 23 14:59:48.748807 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 14:59:48.750352 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 14:59:48.750352 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 14:59:48.750352 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Sep 23 14:59:48.750352 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 401/836] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Sep 23 14:59:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 23 14:59:48.754024 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65688afb-3dd1-4c71-899c-e65b07616495 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:48.784310 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:48.811889 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 436/837] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Sep 23 14:59:48 2022] POST /identity/v3/endpoints => generated 346 bytes in 315 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 23 14:59:48.880808 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65688afb-3dd1-4c71-899c-e65b07616495 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:48.930282 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e98348-4fdb-43a6-9631-0866387d7b1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:48.931441 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e98348-4fdb-43a6-9631-0866387d7b1f None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:48.931915 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e98348-4fdb-43a6-9631-0866387d7b1f None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:48.932248 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e98348-4fdb-43a6-9631-0866387d7b1f None admin] PATH_INFO: `/v3/domains` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:48.933137 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e98348-4fdb-43a6-9631-0866387d7b1f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:59:48.933575 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e98348-4fdb-43a6-9631-0866387d7b1f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:48.934994 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e98348-4fdb-43a6-9631-0866387d7b1f None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:48.953946 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 402/838] 10.222.0.55 () {56 vars in 1109 bytes} [Fri Sep 23 14:59:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 14:59:48.959835 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da1a5a0-f23a-48d2-9b72-3e06600ea334 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:49.081409 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:49.091118 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da1a5a0-f23a-48d2-9b72-3e06600ea334 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:49.129962 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0913fc56-8135-4a5c-a55b-c65b04aef076 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:49.130726 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0913fc56-8135-4a5c-a55b-c65b04aef076 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:49.130905 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0913fc56-8135-4a5c-a55b-c65b04aef076 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:49.131214 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0913fc56-8135-4a5c-a55b-c65b04aef076 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:49.140286 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0913fc56-8135-4a5c-a55b-c65b04aef076 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:49.141770 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0913fc56-8135-4a5c-a55b-c65b04aef076 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:49.154597 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-0913fc56-8135-4a5c-a55b-c65b04aef076 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 23 14:59:49.154597 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 14:59:49.154597 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 14:59:49.154597 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 14:59:49.154597 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 14:59:49.154597 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 14:59:49.154597 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 14:59:49.154597 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 14:59:49.154597 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 14:59:49.154597 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 14:59:49.154597 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 14:59:49.154597 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 14:59:49.154597 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 23 14:59:49.154597 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 23 14:59:49.154597 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 23 14:59:49.154597 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 23 14:59:49.157011 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 14:59:49.157011 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 14:59:49.157011 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 23 14:59:49.157011 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 14:59:49.157011 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 23 14:59:49.157011 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 14:59:49.157011 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 14:59:49.157011 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 14:59:49.157011 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 14:59:49.157011 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 14:59:49.157011 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 14:59:49.157011 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Sep 23 14:59:49.157011 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 14:59:49.157011 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Sep 23 14:59:49.157011 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 14:59:49.157011 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 14:59:49.157011 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 14:59:49.157011 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Sep 23 14:59:49.159606 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 14:59:49.159606 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Sep 23 14:59:49.159606 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 23 14:59:49.159606 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 23 14:59:49.159606 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 23 14:59:49.159606 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 23 14:59:49.159606 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 23 14:59:49.159606 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 23 14:59:49.159606 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 23 14:59:49.159606 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Sep 23 14:59:49.162680 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 437/839] 10.222.0.55 () {56 vars in 1096 bytes} [Fri Sep 23 14:59:48 2022] GET /identity/v3/users/swift => generated 83 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 14:59:49.162843 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e98348-4fdb-43a6-9631-0866387d7b1f None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:49.257675 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:49.277595 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e98348-4fdb-43a6-9631-0866387d7b1f None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:49.313310 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f47251-417b-4005-9a76-da9f3a4480e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:49.313919 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f47251-417b-4005-9a76-da9f3a4480e1 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:49.314191 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f47251-417b-4005-9a76-da9f3a4480e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:49.314449 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f47251-417b-4005-9a76-da9f3a4480e1 None admin] PATH_INFO: `/v3/users` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:49.315174 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f47251-417b-4005-9a76-da9f3a4480e1 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:59:49.315469 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f47251-417b-4005-9a76-da9f3a4480e1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:49.316915 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f47251-417b-4005-9a76-da9f3a4480e1 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:49.335332 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 403/840] 10.222.0.55 () {56 vars in 1135 bytes} [Fri Sep 23 14:59:49 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:59:49.341751 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0913fc56-8135-4a5c-a55b-c65b04aef076 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:49.454214 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0913fc56-8135-4a5c-a55b-c65b04aef076 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:49.492134 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266b5815-9bc7-40c9-89d1-0c1451b934cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:49.492755 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266b5815-9bc7-40c9-89d1-0c1451b934cc None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:49.493037 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266b5815-9bc7-40c9-89d1-0c1451b934cc None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:49.493452 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266b5815-9bc7-40c9-89d1-0c1451b934cc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:49.509281 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:49.567635 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266b5815-9bc7-40c9-89d1-0c1451b934cc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:49.569249 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266b5815-9bc7-40c9-89d1-0c1451b934cc None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:49.677964 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 438/841] 10.222.0.55 () {58 vars in 1282 bytes} [Fri Sep 23 14:59:49 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 23 14:59:49.684190 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f47251-417b-4005-9a76-da9f3a4480e1 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:49.798046 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f47251-417b-4005-9a76-da9f3a4480e1 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:49.842901 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e7f8be-90a9-450e-a982-22a772ef13c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:49.844124 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e7f8be-90a9-450e-a982-22a772ef13c4 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:49.844737 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e7f8be-90a9-450e-a982-22a772ef13c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:49.845226 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e7f8be-90a9-450e-a982-22a772ef13c4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:49.859777 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:49.862856 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e7f8be-90a9-450e-a982-22a772ef13c4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:49.864610 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e7f8be-90a9-450e-a982-22a772ef13c4 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:49.880493 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application [None req-34e7f8be-90a9-450e-a982-22a772ef13c4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 14:59:49.880493 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 14:59:49.880493 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 14:59:49.880493 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 14:59:49.880493 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 14:59:49.880493 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 14:59:49.880493 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 14:59:49.880493 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 14:59:49.880493 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 14:59:49.880493 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 14:59:49.880493 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 14:59:49.880493 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 14:59:49.880493 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 14:59:49.880493 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 14:59:49.880493 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 14:59:49.880493 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 14:59:49.881488 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 14:59:49.881488 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 14:59:49.881488 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 14:59:49.881488 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 14:59:49.881488 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 14:59:49.881488 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 14:59:49.881488 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 14:59:49.881488 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application with Lock( Sep 23 14:59:49.881488 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 14:59:49.881488 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._enter() Sep 23 14:59:49.881488 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 14:59:49.881488 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 14:59:49.881488 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 14:59:49.881488 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.creator() Sep 23 14:59:49.881488 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 14:59:49.881488 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application created_value = creator( Sep 23 14:59:49.881488 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 14:59:49.882831 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 14:59:49.882831 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 14:59:49.882831 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Sep 23 14:59:49.885536 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 404/842] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Sep 23 14:59:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 14:59:49.893028 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266b5815-9bc7-40c9-89d1-0c1451b934cc None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:49.940888 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:50.000586 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266b5815-9bc7-40c9-89d1-0c1451b934cc None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:50.037725 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6d918a-0c6a-487e-ad24-719749f057da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:50.038350 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6d918a-0c6a-487e-ad24-719749f057da None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:50.038634 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6d918a-0c6a-487e-ad24-719749f057da None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:50.038903 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6d918a-0c6a-487e-ad24-719749f057da None admin] PATH_INFO: `/v3/domains` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:50.039549 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6d918a-0c6a-487e-ad24-719749f057da None admin] RBAC: Adding query filter params (name=Default) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:59:50.039851 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6d918a-0c6a-487e-ad24-719749f057da None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:50.041684 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6d918a-0c6a-487e-ad24-719749f057da None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:50.053746 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 439/843] 10.222.0.55 () {56 vars in 1109 bytes} [Fri Sep 23 14:59:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:59:50.060436 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e7f8be-90a9-450e-a982-22a772ef13c4 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:50.177080 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e7f8be-90a9-450e-a982-22a772ef13c4 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:50.214887 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16dc62c3-8bc2-4e4b-8e0d-0dea03d830d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:50.215760 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dc62c3-8bc2-4e4b-8e0d-0dea03d830d2 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:50.216289 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dc62c3-8bc2-4e4b-8e0d-0dea03d830d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:50.216708 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dc62c3-8bc2-4e4b-8e0d-0dea03d830d2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:50.224092 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16dc62c3-8bc2-4e4b-8e0d-0dea03d830d2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:50.225787 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16dc62c3-8bc2-4e4b-8e0d-0dea03d830d2 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:50.237848 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application [None req-16dc62c3-8bc2-4e4b-8e0d-0dea03d830d2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 14:59:50.237848 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 14:59:50.237848 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 14:59:50.237848 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 14:59:50.237848 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 14:59:50.237848 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 14:59:50.237848 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 14:59:50.237848 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 14:59:50.237848 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 14:59:50.237848 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 14:59:50.237848 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 14:59:50.237848 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 14:59:50.237848 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 23 14:59:50.237848 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 23 14:59:50.237848 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 23 14:59:50.238995 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 23 14:59:50.238995 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 14:59:50.238995 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 14:59:50.238995 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 14:59:50.238995 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 14:59:50.238995 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 14:59:50.238995 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 14:59:50.238995 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 14:59:50.238995 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application with Lock( Sep 23 14:59:50.238995 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 14:59:50.238995 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._enter() Sep 23 14:59:50.238995 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 14:59:50.238995 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 14:59:50.238995 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 14:59:50.238995 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.creator() Sep 23 14:59:50.238995 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 14:59:50.238995 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application created_value = creator( Sep 23 14:59:50.240128 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 23 14:59:50.240128 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 23 14:59:50.240128 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 23 14:59:50.240128 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 23 14:59:50.240128 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 23 14:59:50.240128 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 23 14:59:50.240128 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 14:59:50.240128 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Sep 23 14:59:50.242118 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 405/844] 10.222.0.55 () {56 vars in 1111 bytes} [Fri Sep 23 14:59:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 14:59:50.248678 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6d918a-0c6a-487e-ad24-719749f057da None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:50.358335 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6d918a-0c6a-487e-ad24-719749f057da None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:50.368932 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:50.381716 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f96b7be-1411-44f5-9a08-fecb1390e188 None None] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:50.387844 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f96b7be-1411-44f5-9a08-fecb1390e188 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:50.387844 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f96b7be-1411-44f5-9a08-fecb1390e188 None None] PATH_INFO: `/` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:50.387844 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 406/845] 10.222.0.55 () {54 vars in 927 bytes} [Fri Sep 23 14:59:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 14:59:50.392952 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf04206b-32c5-4a7e-9b22-14b144f83ff0 None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:50.393772 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf04206b-32c5-4a7e-9b22-14b144f83ff0 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:50.394515 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf04206b-32c5-4a7e-9b22-14b144f83ff0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:50.398054 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4fc0f4-adf7-4a9d-9215-c1cb798efb3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:50.398661 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4fc0f4-adf7-4a9d-9215-c1cb798efb3e None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:50.398939 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4fc0f4-adf7-4a9d-9215-c1cb798efb3e None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:50.399207 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4fc0f4-adf7-4a9d-9215-c1cb798efb3e None admin] PATH_INFO: `/v3/projects` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:50.399955 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4fc0f4-adf7-4a9d-9215-c1cb798efb3e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:59:50.400286 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4fc0f4-adf7-4a9d-9215-c1cb798efb3e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:50.401799 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4fc0f4-adf7-4a9d-9215-c1cb798efb3e None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:50.417238 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 440/846] 10.222.0.55 () {56 vars in 1148 bytes} [Fri Sep 23 14:59:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:59:50.424416 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4fc0f4-adf7-4a9d-9215-c1cb798efb3e None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:50.424581 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-bf04206b-32c5-4a7e-9b22-14b144f83ff0 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 14:59:50.425841 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-bf04206b-32c5-4a7e-9b22-14b144f83ff0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 14:59:50.482542 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:50.512185 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 407/847] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Sep 23 14:59:50 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 23 14:59:50.531620 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4fc0f4-adf7-4a9d-9215-c1cb798efb3e None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:50.571305 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e86a8c5-5f3f-488b-8202-063ed74d841a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:50.572710 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e86a8c5-5f3f-488b-8202-063ed74d841a None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:50.573212 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e86a8c5-5f3f-488b-8202-063ed74d841a None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:50.573687 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e86a8c5-5f3f-488b-8202-063ed74d841a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:50.574611 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e86a8c5-5f3f-488b-8202-063ed74d841a None admin] RBAC: Adding query filter params (role.id=8e2919e79df040e8926f21607818b58c, scope.project.id=87b3e9d4b6474bd4a04ba59ef52bf3e2, user.id=2228c0b49a0249bbaf524a8c4e9e108b) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:59:50.575156 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e86a8c5-5f3f-488b-8202-063ed74d841a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:50.576801 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e86a8c5-5f3f-488b-8202-063ed74d841a None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:50.583595 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 441/848] 10.222.0.55 () {56 vars in 1374 bytes} [Fri Sep 23 14:59:50 2022] GET /identity/v3/role_assignments?user.id=2228c0b49a0249bbaf524a8c4e9e108b&scope.project.id=87b3e9d4b6474bd4a04ba59ef52bf3e2&role.id=8e2919e79df040e8926f21607818b58c => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 14:59:50.788798 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ed923f-ca85-4b4f-90c5-4a276deecb11 None None] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:50.789532 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ed923f-ca85-4b4f-90c5-4a276deecb11 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:50.790065 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ed923f-ca85-4b4f-90c5-4a276deecb11 None None] PATH_INFO: `/` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:50.792678 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 408/849] 10.222.0.55 () {54 vars in 927 bytes} [Fri Sep 23 14:59:50 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 14:59:50.797994 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ab431b-2cb3-4a9c-8571-527af3787bbf None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:50.798276 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ab431b-2cb3-4a9c-8571-527af3787bbf None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:50.798549 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ab431b-2cb3-4a9c-8571-527af3787bbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:50.800948 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ed923f-ca85-4b4f-90c5-4a276deecb11 None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:50.818327 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-27ab431b-2cb3-4a9c-8571-527af3787bbf None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 14:59:50.819404 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-27ab431b-2cb3-4a9c-8571-527af3787bbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 14:59:50.848424 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:50.903936 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 442/850] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Sep 23 14:59:50 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 23 14:59:50.925049 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ed923f-ca85-4b4f-90c5-4a276deecb11 None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:50.961616 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480593e4-7820-4aef-bc3f-ab4c3a4386ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:50.963282 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480593e4-7820-4aef-bc3f-ab4c3a4386ea None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:50.963776 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480593e4-7820-4aef-bc3f-ab4c3a4386ea None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:50.964329 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480593e4-7820-4aef-bc3f-ab4c3a4386ea None admin] PATH_INFO: `/v3/roles/service` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:50.975524 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480593e4-7820-4aef-bc3f-ab4c3a4386ea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 14:59:50.981304 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application [None req-480593e4-7820-4aef-bc3f-ab4c3a4386ea None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 23 14:59:50.981304 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 14:59:50.981304 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 14:59:50.981304 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 14:59:50.981304 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 14:59:50.981304 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 14:59:50.981304 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 14:59:50.981304 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 14:59:50.981304 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 14:59:50.981304 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 14:59:50.981304 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 14:59:50.981304 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 14:59:50.981304 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 14:59:50.981304 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 14:59:50.981304 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 14:59:50.981304 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 14:59:50.982223 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 14:59:50.982223 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 14:59:50.982223 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 14:59:50.982223 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 14:59:50.982223 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 14:59:50.982223 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 14:59:50.982223 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 14:59:50.982223 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 14:59:50.982223 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 14:59:50.982223 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 14:59:50.982223 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 14:59:50.982223 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application with Lock( Sep 23 14:59:50.982223 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 14:59:50.982223 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._enter() Sep 23 14:59:50.982223 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 14:59:50.982223 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 14:59:50.983111 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 14:59:50.983111 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.creator() Sep 23 14:59:50.983111 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 14:59:50.983111 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application created_value = creator( Sep 23 14:59:50.983111 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 14:59:50.983111 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 14:59:50.983111 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 14:59:50.983111 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 14:59:50.983111 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 14:59:50.983111 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 14:59:50.983111 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 23 14:59:50.983111 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Sep 23 14:59:50.985590 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 409/851] 10.222.0.55 () {56 vars in 1102 bytes} [Fri Sep 23 14:59:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 14:59:50.991682 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ab431b-2cb3-4a9c-8571-527af3787bbf None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:51.120476 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ab431b-2cb3-4a9c-8571-527af3787bbf None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:51.124058 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e64ea1-79e9-4aa7-998b-3c09bd3702da None None] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:51.124351 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e64ea1-79e9-4aa7-998b-3c09bd3702da None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:51.124624 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e64ea1-79e9-4aa7-998b-3c09bd3702da None None] PATH_INFO: `/` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:51.126770 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 410/852] 10.222.0.55 () {54 vars in 927 bytes} [Fri Sep 23 14:59:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 23 14:59:51.131916 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39874e0b-d8ac-4b12-aa97-a9d752e6a83d None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:51.132265 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39874e0b-d8ac-4b12-aa97-a9d752e6a83d None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:51.132541 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39874e0b-d8ac-4b12-aa97-a9d752e6a83d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:51.152446 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-39874e0b-d8ac-4b12-aa97-a9d752e6a83d None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 14:59:51.153905 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-39874e0b-d8ac-4b12-aa97-a9d752e6a83d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 14:59:51.164843 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783fe9cc-5113-4972-a944-ef8a374db1be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:51.166352 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783fe9cc-5113-4972-a944-ef8a374db1be None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:51.167074 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783fe9cc-5113-4972-a944-ef8a374db1be None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:51.167543 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783fe9cc-5113-4972-a944-ef8a374db1be None admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:51.168686 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783fe9cc-5113-4972-a944-ef8a374db1be None admin] RBAC: Adding query filter params (name=service) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:59:51.169188 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783fe9cc-5113-4972-a944-ef8a374db1be None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:51.171447 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783fe9cc-5113-4972-a944-ef8a374db1be None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:51.182843 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 443/853] 10.222.0.55 () {56 vars in 1103 bytes} [Fri Sep 23 14:59:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:59:51.189010 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783fe9cc-5113-4972-a944-ef8a374db1be None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:51.195507 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:51.216521 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:51.233691 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 411/854] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Sep 23 14:59:51 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 23 14:59:51.234605 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39874e0b-d8ac-4b12-aa97-a9d752e6a83d None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:51.307009 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783fe9cc-5113-4972-a944-ef8a374db1be None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:51.345475 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1793b47-9d5a-4bfe-a047-e587f68673b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:51.346032 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1793b47-9d5a-4bfe-a047-e587f68673b6 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:51.346312 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1793b47-9d5a-4bfe-a047-e587f68673b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:51.346587 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1793b47-9d5a-4bfe-a047-e587f68673b6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:51.347213 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39874e0b-d8ac-4b12-aa97-a9d752e6a83d None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:51.358373 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1793b47-9d5a-4bfe-a047-e587f68673b6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:51.360109 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1793b47-9d5a-4bfe-a047-e587f68673b6 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:51.376694 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-b1793b47-9d5a-4bfe-a047-e587f68673b6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 14:59:51.376694 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 14:59:51.376694 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 14:59:51.376694 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 14:59:51.376694 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 14:59:51.376694 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 14:59:51.376694 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 14:59:51.376694 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 14:59:51.376694 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 14:59:51.376694 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 14:59:51.376694 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 14:59:51.376694 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 14:59:51.376694 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 14:59:51.376694 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 14:59:51.376694 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 14:59:51.376694 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 14:59:51.377711 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 14:59:51.377711 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 14:59:51.377711 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 14:59:51.377711 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 14:59:51.377711 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 14:59:51.377711 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 14:59:51.377711 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 14:59:51.377711 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Sep 23 14:59:51.377711 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 14:59:51.377711 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Sep 23 14:59:51.377711 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 14:59:51.377711 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 14:59:51.377711 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 14:59:51.377711 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Sep 23 14:59:51.377711 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 14:59:51.377711 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Sep 23 14:59:51.377711 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 14:59:51.378974 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 14:59:51.378974 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 14:59:51.378974 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Sep 23 14:59:51.378974 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 444/855] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Sep 23 14:59:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 23 14:59:51.385377 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1793b47-9d5a-4bfe-a047-e587f68673b6 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:51.392494 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1978f3cf-2c75-40ce-8368-e8e7e0e7e1e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:51.393016 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1978f3cf-2c75-40ce-8368-e8e7e0e7e1e2 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:51.393275 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1978f3cf-2c75-40ce-8368-e8e7e0e7e1e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:51.393546 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1978f3cf-2c75-40ce-8368-e8e7e0e7e1e2 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:51.394131 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1978f3cf-2c75-40ce-8368-e8e7e0e7e1e2 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:51.395505 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1978f3cf-2c75-40ce-8368-e8e7e0e7e1e2 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:51.406368 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application [None req-1978f3cf-2c75-40ce-8368-e8e7e0e7e1e2 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 23 14:59:51.406368 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 14:59:51.406368 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 23 14:59:51.406368 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 23 14:59:51.406368 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 23 14:59:51.406368 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 23 14:59:51.406368 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 23 14:59:51.406368 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 23 14:59:51.406368 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 23 14:59:51.406368 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Sep 23 14:59:51.406368 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 23 14:59:51.406368 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Sep 23 14:59:51.406368 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 14:59:51.406368 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 14:59:51.406368 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 14:59:51.406368 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 14:59:51.407392 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 14:59:51.407392 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 14:59:51.407392 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 14:59:51.407392 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 14:59:51.407392 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 14:59:51.407392 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 14:59:51.407392 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 14:59:51.407392 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 23 14:59:51.407392 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 23 14:59:51.407392 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 23 14:59:51.407392 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 23 14:59:51.407392 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 14:59:51.407392 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 14:59:51.407392 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 14:59:51.407392 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 14:59:51.407392 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 14:59:51.407392 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 14:59:51.415156 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 14:59:51.415156 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application with Lock( Sep 23 14:59:51.415156 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 14:59:51.415156 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._enter() Sep 23 14:59:51.415156 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 14:59:51.415156 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 14:59:51.415156 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 14:59:51.415156 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.creator() Sep 23 14:59:51.415156 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 14:59:51.415156 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application created_value = creator( Sep 23 14:59:51.415156 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 23 14:59:51.415156 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 23 14:59:51.415156 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 23 14:59:51.415156 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Sep 23 14:59:51.415156 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 412/856] 10.222.0.55 () {56 vars in 1132 bytes} [Fri Sep 23 14:59:51 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 14:59:51.418112 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1978f3cf-2c75-40ce-8368-e8e7e0e7e1e2 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:51.459959 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:51.483315 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:51.520224 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1793b47-9d5a-4bfe-a047-e587f68673b6 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:51.542901 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1978f3cf-2c75-40ce-8368-e8e7e0e7e1e2 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:51.559492 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179f1082-00a4-4056-8af5-cbd661d956ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:51.560156 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179f1082-00a4-4056-8af5-cbd661d956ec None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:51.560477 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179f1082-00a4-4056-8af5-cbd661d956ec None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:51.560707 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179f1082-00a4-4056-8af5-cbd661d956ec None admin] PATH_INFO: `/v3/domains` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:51.561352 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179f1082-00a4-4056-8af5-cbd661d956ec None admin] RBAC: Adding query filter params (name=Default) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:59:51.561656 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179f1082-00a4-4056-8af5-cbd661d956ec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:51.563017 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179f1082-00a4-4056-8af5-cbd661d956ec None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:51.575843 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 445/857] 10.222.0.55 () {56 vars in 1109 bytes} [Fri Sep 23 14:59:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:59:51.579413 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533d8bc2-ee44-426e-a912-5630dbc1f90e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:51.580099 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533d8bc2-ee44-426e-a912-5630dbc1f90e None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:51.580588 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533d8bc2-ee44-426e-a912-5630dbc1f90e None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:51.580867 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533d8bc2-ee44-426e-a912-5630dbc1f90e None admin] PATH_INFO: `/v3/services` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:51.581094 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179f1082-00a4-4056-8af5-cbd661d956ec None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:51.581505 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533d8bc2-ee44-426e-a912-5630dbc1f90e None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:59:51.581902 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533d8bc2-ee44-426e-a912-5630dbc1f90e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:51.583244 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533d8bc2-ee44-426e-a912-5630dbc1f90e None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:51.593668 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 413/858] 10.222.0.55 () {56 vars in 1126 bytes} [Fri Sep 23 14:59:51 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 14:59:51.593668 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533d8bc2-ee44-426e-a912-5630dbc1f90e None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:51.692637 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:51.695365 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533d8bc2-ee44-426e-a912-5630dbc1f90e None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:51.704120 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179f1082-00a4-4056-8af5-cbd661d956ec None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:51.733673 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54615059-95a2-4f61-9366-68feea407403 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:51.734303 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54615059-95a2-4f61-9366-68feea407403 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:51.734579 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54615059-95a2-4f61-9366-68feea407403 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:51.734848 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54615059-95a2-4f61-9366-68feea407403 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:51.742941 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54615059-95a2-4f61-9366-68feea407403 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:51.744100 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:51.745203 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54615059-95a2-4f61-9366-68feea407403 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:51.749819 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc9941a-f38b-4960-b9d7-e7bf0bb079e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:51.750706 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc9941a-f38b-4960-b9d7-e7bf0bb079e5 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:51.751184 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc9941a-f38b-4960-b9d7-e7bf0bb079e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:51.751655 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc9941a-f38b-4960-b9d7-e7bf0bb079e5 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:51.752473 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc9941a-f38b-4960-b9d7-e7bf0bb079e5 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:51.754041 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc9941a-f38b-4960-b9d7-e7bf0bb079e5 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:51.757335 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application [None req-54615059-95a2-4f61-9366-68feea407403 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 23 14:59:51.757335 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 14:59:51.757335 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 14:59:51.757335 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 14:59:51.757335 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 14:59:51.757335 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 14:59:51.757335 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 14:59:51.757335 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 14:59:51.757335 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 14:59:51.757335 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 14:59:51.757335 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 14:59:51.757335 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 14:59:51.757335 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 23 14:59:51.757335 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 23 14:59:51.757335 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 23 14:59:51.757335 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 23 14:59:51.758290 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 14:59:51.758290 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 14:59:51.758290 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 23 14:59:51.758290 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 14:59:51.758290 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 23 14:59:51.758290 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 14:59:51.758290 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 14:59:51.758290 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 14:59:51.758290 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 14:59:51.758290 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 14:59:51.758290 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 14:59:51.758290 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application with Lock( Sep 23 14:59:51.758290 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 14:59:51.758290 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._enter() Sep 23 14:59:51.758290 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 14:59:51.758290 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 14:59:51.758290 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 14:59:51.758290 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.creator() Sep 23 14:59:51.760091 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 14:59:51.760091 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application created_value = creator( Sep 23 14:59:51.760091 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 23 14:59:51.760091 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 23 14:59:51.760091 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 23 14:59:51.760091 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 23 14:59:51.760091 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 23 14:59:51.760091 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 23 14:59:51.760091 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 23 14:59:51.760091 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Sep 23 14:59:51.760091 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 414/859] 10.222.0.55 () {56 vars in 1117 bytes} [Fri Sep 23 14:59:51 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 14:59:51.760091 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54615059-95a2-4f61-9366-68feea407403 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:51.770640 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-edc9941a-f38b-4960-b9d7-e7bf0bb079e5 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 23 14:59:51.770640 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 14:59:51.770640 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 23 14:59:51.770640 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 23 14:59:51.770640 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 23 14:59:51.770640 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 23 14:59:51.770640 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 23 14:59:51.770640 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 23 14:59:51.770640 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 23 14:59:51.770640 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Sep 23 14:59:51.770640 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 23 14:59:51.770640 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Sep 23 14:59:51.770640 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 14:59:51.770640 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 14:59:51.770640 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 14:59:51.770640 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 14:59:51.770640 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 14:59:51.771620 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 14:59:51.771620 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 14:59:51.771620 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 14:59:51.771620 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 14:59:51.771620 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 14:59:51.771620 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 14:59:51.771620 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 23 14:59:51.771620 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 23 14:59:51.771620 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 23 14:59:51.771620 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 23 14:59:51.771620 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 14:59:51.771620 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 14:59:51.771620 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 14:59:51.771620 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 14:59:51.771620 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 14:59:51.771620 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 14:59:51.772470 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 14:59:51.772470 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Sep 23 14:59:51.772470 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 14:59:51.772470 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Sep 23 14:59:51.772470 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 14:59:51.772470 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 14:59:51.772470 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 14:59:51.772470 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Sep 23 14:59:51.772470 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 14:59:51.772470 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Sep 23 14:59:51.772470 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 23 14:59:51.772470 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 23 14:59:51.772470 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 23 14:59:51.772470 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Sep 23 14:59:51.773298 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 446/860] 10.222.0.55 () {56 vars in 1114 bytes} [Fri Sep 23 14:59:51 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 14:59:51.774348 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc9941a-f38b-4960-b9d7-e7bf0bb079e5 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:51.873286 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54615059-95a2-4f61-9366-68feea407403 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:51.883179 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc9941a-f38b-4960-b9d7-e7bf0bb079e5 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:51.912790 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f14f85e-4812-4b96-945f-ae802db45e9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:51.913399 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f14f85e-4812-4b96-945f-ae802db45e9a None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:51.913676 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f14f85e-4812-4b96-945f-ae802db45e9a None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:51.913944 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f14f85e-4812-4b96-945f-ae802db45e9a None admin] PATH_INFO: `/v3/services` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:51.914565 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f14f85e-4812-4b96-945f-ae802db45e9a None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:59:51.914864 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f14f85e-4812-4b96-945f-ae802db45e9a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:51.916996 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f14f85e-4812-4b96-945f-ae802db45e9a None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:51.918526 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d173537d-f47e-4e2f-a86d-806674bad4d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:51.919135 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d173537d-f47e-4e2f-a86d-806674bad4d8 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:51.919419 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d173537d-f47e-4e2f-a86d-806674bad4d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:51.919697 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d173537d-f47e-4e2f-a86d-806674bad4d8 None admin] PATH_INFO: `/v3/users` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:51.920463 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d173537d-f47e-4e2f-a86d-806674bad4d8 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:59:51.920772 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d173537d-f47e-4e2f-a86d-806674bad4d8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:51.928299 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 415/861] 10.222.0.55 () {56 vars in 1126 bytes} [Fri Sep 23 14:59:51 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:59:51.928488 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d173537d-f47e-4e2f-a86d-806674bad4d8 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:51.928592 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f14f85e-4812-4b96-945f-ae802db45e9a None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:51.948385 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 447/862] 10.222.0.55 () {56 vars in 1149 bytes} [Fri Sep 23 14:59:51 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:59:51.949731 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d173537d-f47e-4e2f-a86d-806674bad4d8 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:51.968444 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:52.029665 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:52.078149 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f14f85e-4812-4b96-945f-ae802db45e9a None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:52.093488 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d173537d-f47e-4e2f-a86d-806674bad4d8 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:52.123322 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd41f8bb-afce-4ddc-b721-5a607eee8c1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:52.124872 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd41f8bb-afce-4ddc-b721-5a607eee8c1a None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:52.125341 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd41f8bb-afce-4ddc-b721-5a607eee8c1a None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:52.125699 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd41f8bb-afce-4ddc-b721-5a607eee8c1a None admin] PATH_INFO: `/v3/services` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:52.126709 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd41f8bb-afce-4ddc-b721-5a607eee8c1a None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:59:52.127441 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd41f8bb-afce-4ddc-b721-5a607eee8c1a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:52.129258 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd41f8bb-afce-4ddc-b721-5a607eee8c1a None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:52.138397 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6875ed2f-e69b-4934-a12b-86d75086cd78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:52.139063 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6875ed2f-e69b-4934-a12b-86d75086cd78 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:52.139349 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6875ed2f-e69b-4934-a12b-86d75086cd78 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:52.139620 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6875ed2f-e69b-4934-a12b-86d75086cd78 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:52.140098 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 416/863] 10.222.0.55 () {56 vars in 1114 bytes} [Fri Sep 23 14:59:51 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:59:52.142322 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd41f8bb-afce-4ddc-b721-5a607eee8c1a None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:52.222071 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6875ed2f-e69b-4934-a12b-86d75086cd78 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:52.223862 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6875ed2f-e69b-4934-a12b-86d75086cd78 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:52.252762 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:52.268882 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd41f8bb-afce-4ddc-b721-5a607eee8c1a None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:52.298602 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:52.315540 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9158158d-a624-4d4a-aeb9-63b2665693b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:52.316335 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9158158d-a624-4d4a-aeb9-63b2665693b0 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:52.316716 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9158158d-a624-4d4a-aeb9-63b2665693b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:52.317090 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9158158d-a624-4d4a-aeb9-63b2665693b0 None admin] PATH_INFO: `/v3/services` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:52.317829 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9158158d-a624-4d4a-aeb9-63b2665693b0 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:59:52.318238 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9158158d-a624-4d4a-aeb9-63b2665693b0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:52.319715 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9158158d-a624-4d4a-aeb9-63b2665693b0 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:52.327090 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 417/864] 10.222.0.55 () {56 vars in 1114 bytes} [Fri Sep 23 14:59:52 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:59:52.361072 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 448/865] 10.222.0.55 () {58 vars in 1282 bytes} [Fri Sep 23 14:59:51 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 23 14:59:52.366774 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9158158d-a624-4d4a-aeb9-63b2665693b0 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:52.489499 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9158158d-a624-4d4a-aeb9-63b2665693b0 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:52.530431 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ada13d-210f-46e8-bbb5-8798231fc732 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:52.531088 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ada13d-210f-46e8-bbb5-8798231fc732 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:52.531368 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ada13d-210f-46e8-bbb5-8798231fc732 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:52.531643 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ada13d-210f-46e8-bbb5-8798231fc732 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:52.543949 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ada13d-210f-46e8-bbb5-8798231fc732 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:52.545443 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ada13d-210f-46e8-bbb5-8798231fc732 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:52.561511 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application [None req-e9ada13d-210f-46e8-bbb5-8798231fc732 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 14:59:52.561511 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 14:59:52.561511 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 14:59:52.561511 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 14:59:52.561511 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 14:59:52.561511 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 14:59:52.561511 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 14:59:52.561511 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 14:59:52.561511 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 14:59:52.561511 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 14:59:52.561511 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 14:59:52.561511 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 14:59:52.561511 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 14:59:52.561511 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 14:59:52.561511 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 14:59:52.561511 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 14:59:52.562508 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 14:59:52.562508 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 14:59:52.562508 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 14:59:52.562508 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 14:59:52.562508 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 14:59:52.562508 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 14:59:52.562508 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 14:59:52.562508 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application with Lock( Sep 23 14:59:52.562508 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 14:59:52.562508 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._enter() Sep 23 14:59:52.562508 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 14:59:52.562508 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 14:59:52.562508 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 14:59:52.562508 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.creator() Sep 23 14:59:52.562508 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 14:59:52.562508 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application created_value = creator( Sep 23 14:59:52.562508 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 14:59:52.563388 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 14:59:52.563388 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 14:59:52.563388 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Sep 23 14:59:52.567354 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 418/866] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Sep 23 14:59:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 23 14:59:52.568766 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6875ed2f-e69b-4934-a12b-86d75086cd78 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:52.688066 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6875ed2f-e69b-4934-a12b-86d75086cd78 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:52.727320 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ddf155-6184-41d7-80a3-4d4b1b3c8a41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:52.728691 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ddf155-6184-41d7-80a3-4d4b1b3c8a41 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:52.729247 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ddf155-6184-41d7-80a3-4d4b1b3c8a41 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:52.729728 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ddf155-6184-41d7-80a3-4d4b1b3c8a41 None admin] PATH_INFO: `/v3/domains` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:52.730623 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ddf155-6184-41d7-80a3-4d4b1b3c8a41 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:59:52.731133 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ddf155-6184-41d7-80a3-4d4b1b3c8a41 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:52.732781 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ddf155-6184-41d7-80a3-4d4b1b3c8a41 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:52.751721 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 449/867] 10.222.0.55 () {56 vars in 1109 bytes} [Fri Sep 23 14:59:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:59:52.756338 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ada13d-210f-46e8-bbb5-8798231fc732 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:52.760786 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:52.871865 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ada13d-210f-46e8-bbb5-8798231fc732 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:52.911247 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421efbc8-13af-4fe4-8389-a3fd7cb64709 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:52.912280 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421efbc8-13af-4fe4-8389-a3fd7cb64709 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:52.912907 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421efbc8-13af-4fe4-8389-a3fd7cb64709 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:52.913179 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421efbc8-13af-4fe4-8389-a3fd7cb64709 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:52.913497 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff9ea78-8f72-4e90-9945-bbafb4d61f30 None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:52.913801 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff9ea78-8f72-4e90-9945-bbafb4d61f30 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:52.914064 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff9ea78-8f72-4e90-9945-bbafb4d61f30 None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:52.920087 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 450/868] 10.222.0.55 () {54 vars in 927 bytes} [Fri Sep 23 14:59:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 14:59:52.921680 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c386095d-da7e-4ccb-895c-b2ee85782314 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:52.922027 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c386095d-da7e-4ccb-895c-b2ee85782314 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:52.922306 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c386095d-da7e-4ccb-895c-b2ee85782314 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:52.924906 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421efbc8-13af-4fe4-8389-a3fd7cb64709 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:52.926349 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421efbc8-13af-4fe4-8389-a3fd7cb64709 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:52.938043 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application [None req-421efbc8-13af-4fe4-8389-a3fd7cb64709 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 14:59:52.938043 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 14:59:52.938043 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 14:59:52.938043 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 14:59:52.938043 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 14:59:52.938043 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 14:59:52.938043 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 14:59:52.938043 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 14:59:52.938043 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 14:59:52.938043 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 14:59:52.938043 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 14:59:52.938043 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 14:59:52.938043 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 23 14:59:52.938043 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 23 14:59:52.938043 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 23 14:59:52.938996 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 23 14:59:52.938996 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 14:59:52.938996 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 14:59:52.938996 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 14:59:52.938996 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 14:59:52.938996 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 14:59:52.938996 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 14:59:52.938996 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 14:59:52.938996 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application with Lock( Sep 23 14:59:52.938996 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 14:59:52.938996 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._enter() Sep 23 14:59:52.938996 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 14:59:52.938996 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 14:59:52.938996 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 14:59:52.938996 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.creator() Sep 23 14:59:52.938996 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 14:59:52.938996 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application created_value = creator( Sep 23 14:59:52.940016 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 23 14:59:52.940016 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 23 14:59:52.940016 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 23 14:59:52.940016 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 23 14:59:52.940016 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 23 14:59:52.940016 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 23 14:59:52.940016 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 14:59:52.940016 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Sep 23 14:59:52.940016 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 419/869] 10.222.0.55 () {56 vars in 1111 bytes} [Fri Sep 23 14:59:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 14:59:52.945359 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421efbc8-13af-4fe4-8389-a3fd7cb64709 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:52.948097 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-c386095d-da7e-4ccb-895c-b2ee85782314 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 14:59:52.949514 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-c386095d-da7e-4ccb-895c-b2ee85782314 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 14:59:52.972256 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:53.028178 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 451/870] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Sep 23 14:59:52 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 23 14:59:53.035135 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:53.086104 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421efbc8-13af-4fe4-8389-a3fd7cb64709 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:53.120835 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156b9973-3fce-4cb0-b991-71486a020b36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:53.121500 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156b9973-3fce-4cb0-b991-71486a020b36 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:53.121778 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156b9973-3fce-4cb0-b991-71486a020b36 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:53.122050 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156b9973-3fce-4cb0-b991-71486a020b36 None admin] PATH_INFO: `/v3/projects` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:53.122739 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156b9973-3fce-4cb0-b991-71486a020b36 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:59:53.123085 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156b9973-3fce-4cb0-b991-71486a020b36 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:53.124600 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156b9973-3fce-4cb0-b991-71486a020b36 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:53.138268 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 420/871] 10.222.0.55 () {56 vars in 1148 bytes} [Fri Sep 23 14:59:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 14:59:53.149097 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c386095d-da7e-4ccb-895c-b2ee85782314 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:53.256024 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c386095d-da7e-4ccb-895c-b2ee85782314 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:53.294293 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b7f44a-5844-4944-b765-218bc5f668a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:53.295155 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b7f44a-5844-4944-b765-218bc5f668a4 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:53.295645 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b7f44a-5844-4944-b765-218bc5f668a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:53.296126 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b7f44a-5844-4944-b765-218bc5f668a4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:53.297004 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b7f44a-5844-4944-b765-218bc5f668a4 None admin] RBAC: Adding query filter params (role.id=0665869b71fa40b0b552efdd57faf023, scope.project.id=87b3e9d4b6474bd4a04ba59ef52bf3e2, user.id=68206a6b4f244d45bf7bf1504396e430) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:59:53.297466 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b7f44a-5844-4944-b765-218bc5f668a4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:53.299055 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b7f44a-5844-4944-b765-218bc5f668a4 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:53.310586 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 452/872] 10.222.0.55 () {56 vars in 1374 bytes} [Fri Sep 23 14:59:53 2022] GET /identity/v3/role_assignments?user.id=68206a6b4f244d45bf7bf1504396e430&scope.project.id=87b3e9d4b6474bd4a04ba59ef52bf3e2&role.id=0665869b71fa40b0b552efdd57faf023 => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:59:53.339568 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156b9973-3fce-4cb0-b991-71486a020b36 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:53.463933 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156b9973-3fce-4cb0-b991-71486a020b36 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:53.491057 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:53.502991 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fe9b35-d8b4-4e2a-ab42-d45de264a52b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:53.503633 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fe9b35-d8b4-4e2a-ab42-d45de264a52b None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:53.503913 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fe9b35-d8b4-4e2a-ab42-d45de264a52b None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:53.504217 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fe9b35-d8b4-4e2a-ab42-d45de264a52b None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:53.504806 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fe9b35-d8b4-4e2a-ab42-d45de264a52b None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:53.506168 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fe9b35-d8b4-4e2a-ab42-d45de264a52b None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:53.516104 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application [None req-a1fe9b35-d8b4-4e2a-ab42-d45de264a52b None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 23 14:59:53.516104 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 14:59:53.516104 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 23 14:59:53.516104 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 23 14:59:53.516104 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 23 14:59:53.516104 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 23 14:59:53.516104 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 23 14:59:53.516104 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 23 14:59:53.516104 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 23 14:59:53.516104 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Sep 23 14:59:53.516104 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 23 14:59:53.516104 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Sep 23 14:59:53.516104 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 14:59:53.516104 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 14:59:53.516104 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 14:59:53.516104 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 14:59:53.516104 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 14:59:53.517156 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 14:59:53.517156 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 14:59:53.517156 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 14:59:53.517156 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 14:59:53.517156 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 14:59:53.517156 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 14:59:53.517156 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 23 14:59:53.517156 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 23 14:59:53.517156 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 23 14:59:53.517156 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 23 14:59:53.517156 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 14:59:53.517156 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 14:59:53.517156 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 14:59:53.517156 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 14:59:53.517156 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 14:59:53.517156 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 14:59:53.518018 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 14:59:53.518018 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application with Lock( Sep 23 14:59:53.518018 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 14:59:53.518018 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._enter() Sep 23 14:59:53.518018 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 14:59:53.518018 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 14:59:53.518018 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 14:59:53.518018 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.creator() Sep 23 14:59:53.518018 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 14:59:53.518018 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application created_value = creator( Sep 23 14:59:53.518018 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 23 14:59:53.518018 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 23 14:59:53.518018 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 23 14:59:53.518018 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Sep 23 14:59:53.518018 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 421/873] 10.222.0.55 () {56 vars in 1126 bytes} [Fri Sep 23 14:59:53 2022] GET /identity/v3/services/object-store => generated 93 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 14:59:53.525503 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b7f44a-5844-4944-b765-218bc5f668a4 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:53.539557 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:53.636533 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b7f44a-5844-4944-b765-218bc5f668a4 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:53.674618 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9411edf-dfc6-4a08-86a2-6d28f74b2082 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:53.675506 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9411edf-dfc6-4a08-86a2-6d28f74b2082 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:53.675942 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9411edf-dfc6-4a08-86a2-6d28f74b2082 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:53.676384 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9411edf-dfc6-4a08-86a2-6d28f74b2082 None admin] PATH_INFO: `/v3/services` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:53.677156 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9411edf-dfc6-4a08-86a2-6d28f74b2082 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:59:53.677773 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9411edf-dfc6-4a08-86a2-6d28f74b2082 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:53.679265 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9411edf-dfc6-4a08-86a2-6d28f74b2082 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:53.684831 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 453/874] 10.222.0.55 () {56 vars in 1122 bytes} [Fri Sep 23 14:59:53 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:59:53.690684 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fe9b35-d8b4-4e2a-ab42-d45de264a52b None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:53.799445 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fe9b35-d8b4-4e2a-ab42-d45de264a52b None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:53.834680 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46693b2-409b-497d-a4f2-80363f929995 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:53.835700 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46693b2-409b-497d-a4f2-80363f929995 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:53.836201 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46693b2-409b-497d-a4f2-80363f929995 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:53.836666 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46693b2-409b-497d-a4f2-80363f929995 None admin] PATH_INFO: `/v3/services` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:53.838029 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46693b2-409b-497d-a4f2-80363f929995 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:59:53.838507 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46693b2-409b-497d-a4f2-80363f929995 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:53.840107 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46693b2-409b-497d-a4f2-80363f929995 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:53.845468 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 422/875] 10.222.0.55 () {56 vars in 1122 bytes} [Fri Sep 23 14:59:53 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:59:54.244025 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40badd69-4967-4095-b05c-935b5a5cf490 None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:54.244427 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40badd69-4967-4095-b05c-935b5a5cf490 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:54.244543 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40badd69-4967-4095-b05c-935b5a5cf490 None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:54.246147 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 454/876] 10.222.0.55 () {54 vars in 927 bytes} [Fri Sep 23 14:59:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 14:59:54.252079 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ed1d59-d7ef-45bb-966d-35fd16d6b126 None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:54.252626 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ed1d59-d7ef-45bb-966d-35fd16d6b126 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:54.253115 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ed1d59-d7ef-45bb-966d-35fd16d6b126 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:54.272329 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-b1ed1d59-d7ef-45bb-966d-35fd16d6b126 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 14:59:54.274212 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-b1ed1d59-d7ef-45bb-966d-35fd16d6b126 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 14:59:54.328857 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:54.354307 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 423/877] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Sep 23 14:59:54 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 23 14:59:54.633778 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40badd69-4967-4095-b05c-935b5a5cf490 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:54.720445 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:54.753626 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40badd69-4967-4095-b05c-935b5a5cf490 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:54.771882 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89826885-c642-40de-9f14-736ed8266e9b None None] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:54.772259 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89826885-c642-40de-9f14-736ed8266e9b None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:54.772493 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89826885-c642-40de-9f14-736ed8266e9b None None] PATH_INFO: `/` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:54.774374 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 424/878] 10.222.0.55 () {54 vars in 927 bytes} [Fri Sep 23 14:59:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 14:59:54.780999 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3171f45-c2ad-4c8e-beef-fd00e4a499d3 None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:54.781318 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3171f45-c2ad-4c8e-beef-fd00e4a499d3 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:54.781559 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3171f45-c2ad-4c8e-beef-fd00e4a499d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:54.792428 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6693635-fb8d-4ed5-ab7d-b6bdf80cbfb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:54.796295 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6693635-fb8d-4ed5-ab7d-b6bdf80cbfb0 None admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:54.796295 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6693635-fb8d-4ed5-ab7d-b6bdf80cbfb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:54.796295 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6693635-fb8d-4ed5-ab7d-b6bdf80cbfb0 None admin] PATH_INFO: `/v3/services` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:54.796295 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6693635-fb8d-4ed5-ab7d-b6bdf80cbfb0 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:54.796615 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6693635-fb8d-4ed5-ab7d-b6bdf80cbfb0 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:54.801933 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-e3171f45-c2ad-4c8e-beef-fd00e4a499d3 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 14:59:54.803124 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-e3171f45-c2ad-4c8e-beef-fd00e4a499d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 14:59:54.870853 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 425/879] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Sep 23 14:59:54 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 23 14:59:54.905255 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 455/880] 10.222.0.55 () {60 vars in 1141 bytes} [Fri Sep 23 14:59:54 2022] POST /identity/v3/services => generated 269 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 14:59:55.141604 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3171f45-c2ad-4c8e-beef-fd00e4a499d3 None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:55.246562 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3171f45-c2ad-4c8e-beef-fd00e4a499d3 None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:55.261718 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:55.289010 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a9cfc2-0873-4d31-bf22-6e73fa6fef80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:55.289626 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a9cfc2-0873-4d31-bf22-6e73fa6fef80 None admin] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:55.289902 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a9cfc2-0873-4d31-bf22-6e73fa6fef80 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:55.290282 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a9cfc2-0873-4d31-bf22-6e73fa6fef80 None admin] PATH_INFO: `/v3/services` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:55.291137 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a9cfc2-0873-4d31-bf22-6e73fa6fef80 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:55.292546 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a9cfc2-0873-4d31-bf22-6e73fa6fef80 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:55.453353 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 426/881] 10.222.0.55 () {60 vars in 1141 bytes} [Fri Sep 23 14:59:55 2022] POST /identity/v3/services => generated 251 bytes in 312 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 14:59:55.797258 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40052003-4e25-4343-93f5-3a93a2be6673 None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:55.797778 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40052003-4e25-4343-93f5-3a93a2be6673 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:55.797856 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40052003-4e25-4343-93f5-3a93a2be6673 None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:55.799405 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 456/882] 10.222.0.55 () {54 vars in 927 bytes} [Fri Sep 23 14:59:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 14:59:55.804936 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e327e3-a7c2-44cd-98ac-78f4366cb4c4 None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:55.805230 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e327e3-a7c2-44cd-98ac-78f4366cb4c4 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:55.805509 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e327e3-a7c2-44cd-98ac-78f4366cb4c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:55.827006 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-43e327e3-a7c2-44cd-98ac-78f4366cb4c4 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 14:59:55.828269 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-43e327e3-a7c2-44cd-98ac-78f4366cb4c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 14:59:55.919298 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 427/883] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Sep 23 14:59:55 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 23 14:59:56.102159 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d588d6bb-d336-42e0-8204-7d5f51376e7d None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:56.102547 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d588d6bb-d336-42e0-8204-7d5f51376e7d None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:56.102837 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d588d6bb-d336-42e0-8204-7d5f51376e7d None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:56.104402 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 457/884] 10.222.0.55 () {54 vars in 927 bytes} [Fri Sep 23 14:59:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 14:59:56.109743 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d574147d-16fe-4c6c-a21e-5b5b356d10d1 None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:56.110191 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d574147d-16fe-4c6c-a21e-5b5b356d10d1 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:56.110608 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d574147d-16fe-4c6c-a21e-5b5b356d10d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:56.130508 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-d574147d-16fe-4c6c-a21e-5b5b356d10d1 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 14:59:56.131907 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-d574147d-16fe-4c6c-a21e-5b5b356d10d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 14:59:56.207072 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 428/885] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Sep 23 14:59:56 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 23 14:59:56.223070 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d588d6bb-d336-42e0-8204-7d5f51376e7d None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:56.337151 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d588d6bb-d336-42e0-8204-7d5f51376e7d None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:56.372055 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:56.380365 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144a0fa6-c7bd-4be0-82fa-06732b9317b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:56.381175 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144a0fa6-c7bd-4be0-82fa-06732b9317b9 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:56.381457 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144a0fa6-c7bd-4be0-82fa-06732b9317b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:56.381728 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144a0fa6-c7bd-4be0-82fa-06732b9317b9 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:56.392951 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144a0fa6-c7bd-4be0-82fa-06732b9317b9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 14:59:56.397625 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-144a0fa6-c7bd-4be0-82fa-06732b9317b9 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 23 14:59:56.397625 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 14:59:56.397625 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 14:59:56.397625 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 14:59:56.397625 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 14:59:56.397625 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 14:59:56.397625 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 14:59:56.397625 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 14:59:56.397625 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 14:59:56.397625 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 14:59:56.397625 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 14:59:56.397625 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 14:59:56.397625 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 14:59:56.397625 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 14:59:56.397625 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 14:59:56.397625 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 14:59:56.398527 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 14:59:56.398527 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 14:59:56.398527 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 14:59:56.398527 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 14:59:56.398527 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 14:59:56.398527 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 14:59:56.398527 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 14:59:56.398527 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 14:59:56.398527 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 14:59:56.398527 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 14:59:56.398527 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 14:59:56.398527 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Sep 23 14:59:56.398527 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 14:59:56.398527 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Sep 23 14:59:56.398527 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 14:59:56.398527 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 14:59:56.399382 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 14:59:56.399382 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Sep 23 14:59:56.399382 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 14:59:56.399382 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Sep 23 14:59:56.399382 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 14:59:56.399382 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 14:59:56.399382 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 14:59:56.399382 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 14:59:56.399382 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 14:59:56.399382 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 14:59:56.399382 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 23 14:59:56.399382 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Sep 23 14:59:56.399382 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 458/886] 10.222.0.55 () {56 vars in 1120 bytes} [Fri Sep 23 14:59:56 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 14:59:56.405545 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d574147d-16fe-4c6c-a21e-5b5b356d10d1 None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:56.434842 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:56.483918 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144a0fa6-c7bd-4be0-82fa-06732b9317b9 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:56.521821 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d574147d-16fe-4c6c-a21e-5b5b356d10d1 None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:56.559173 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff03341b-578b-4fa5-a6d6-22e1c2d6f5f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:56.559931 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff03341b-578b-4fa5-a6d6-22e1c2d6f5f5 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:56.560299 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff03341b-578b-4fa5-a6d6-22e1c2d6f5f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:56.560636 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff03341b-578b-4fa5-a6d6-22e1c2d6f5f5 None admin] PATH_INFO: `/v3/roles` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:56.561433 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff03341b-578b-4fa5-a6d6-22e1c2d6f5f5 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:59:56.561758 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff03341b-578b-4fa5-a6d6-22e1c2d6f5f5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:56.563184 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff03341b-578b-4fa5-a6d6-22e1c2d6f5f5 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:56.576834 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 429/887] 10.222.0.55 () {56 vars in 1115 bytes} [Fri Sep 23 14:59:56 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:59:56.577852 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff03341b-578b-4fa5-a6d6-22e1c2d6f5f5 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:56.625698 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144a0fa6-c7bd-4be0-82fa-06732b9317b9 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:56.675191 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc068df-e888-445c-b9fc-7507b7e6445a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:56.676520 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc068df-e888-445c-b9fc-7507b7e6445a None admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:56.677016 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc068df-e888-445c-b9fc-7507b7e6445a None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:56.677890 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:56.678633 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc068df-e888-445c-b9fc-7507b7e6445a None admin] PATH_INFO: `/v3/services` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:56.679512 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc068df-e888-445c-b9fc-7507b7e6445a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:56.682917 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc068df-e888-445c-b9fc-7507b7e6445a None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:56.694622 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff03341b-578b-4fa5-a6d6-22e1c2d6f5f5 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:56.730942 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f062c93-8db6-4b0d-b4ca-a78937ebdd5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:56.731593 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f062c93-8db6-4b0d-b4ca-a78937ebdd5d None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:56.731857 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f062c93-8db6-4b0d-b4ca-a78937ebdd5d None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:56.732164 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f062c93-8db6-4b0d-b4ca-a78937ebdd5d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:56.744627 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f062c93-8db6-4b0d-b4ca-a78937ebdd5d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:56.746123 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f062c93-8db6-4b0d-b4ca-a78937ebdd5d None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:56.762125 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application [None req-7f062c93-8db6-4b0d-b4ca-a78937ebdd5d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 14:59:56.762125 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 14:59:56.762125 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 14:59:56.762125 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 14:59:56.762125 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 14:59:56.762125 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 14:59:56.762125 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 14:59:56.762125 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 14:59:56.762125 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 14:59:56.762125 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 14:59:56.762125 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 14:59:56.762125 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 14:59:56.762125 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 14:59:56.762125 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 14:59:56.762125 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 14:59:56.762125 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 14:59:56.763180 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 14:59:56.763180 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 14:59:56.763180 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 14:59:56.763180 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 14:59:56.763180 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 14:59:56.763180 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 14:59:56.763180 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 14:59:56.763180 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application with Lock( Sep 23 14:59:56.763180 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 14:59:56.763180 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._enter() Sep 23 14:59:56.763180 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 14:59:56.763180 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 14:59:56.763180 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 14:59:56.763180 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.creator() Sep 23 14:59:56.763180 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 14:59:56.763180 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application created_value = creator( Sep 23 14:59:56.763180 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 14:59:56.764141 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 14:59:56.764141 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 14:59:56.764141 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Sep 23 14:59:56.764141 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 430/888] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Sep 23 14:59:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 14:59:56.768705 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f062c93-8db6-4b0d-b4ca-a78937ebdd5d None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:56.808431 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 459/889] 10.222.0.55 () {60 vars in 1141 bytes} [Fri Sep 23 14:59:56 2022] POST /identity/v3/services => generated 241 bytes in 325 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 23 14:59:56.891949 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f062c93-8db6-4b0d-b4ca-a78937ebdd5d None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:56.940515 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdcf2ae-e86c-4469-9c1c-04030e8d6ef8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:56.941525 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdcf2ae-e86c-4469-9c1c-04030e8d6ef8 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:56.942062 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdcf2ae-e86c-4469-9c1c-04030e8d6ef8 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:56.942557 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdcf2ae-e86c-4469-9c1c-04030e8d6ef8 None admin] PATH_INFO: `/v3/domains` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:56.943412 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdcf2ae-e86c-4469-9c1c-04030e8d6ef8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:59:56.943997 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdcf2ae-e86c-4469-9c1c-04030e8d6ef8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:56.945616 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdcf2ae-e86c-4469-9c1c-04030e8d6ef8 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:56.948341 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:56.961625 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 431/890] 10.222.0.55 () {56 vars in 1109 bytes} [Fri Sep 23 14:59:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 14:59:56.969650 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc068df-e888-445c-b9fc-7507b7e6445a None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:57.033447 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:57.088081 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc068df-e888-445c-b9fc-7507b7e6445a None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:57.124359 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7da15a-753b-438d-9183-e9e0a8316666 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:57.125012 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7da15a-753b-438d-9183-e9e0a8316666 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:57.125299 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7da15a-753b-438d-9183-e9e0a8316666 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:57.125579 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7da15a-753b-438d-9183-e9e0a8316666 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:57.134971 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7da15a-753b-438d-9183-e9e0a8316666 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:57.136737 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7da15a-753b-438d-9183-e9e0a8316666 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:57.148788 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-0a7da15a-753b-438d-9183-e9e0a8316666 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 23 14:59:57.148788 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 14:59:57.148788 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 14:59:57.148788 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 14:59:57.148788 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 14:59:57.148788 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 14:59:57.148788 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 14:59:57.148788 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 14:59:57.148788 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 14:59:57.148788 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 14:59:57.148788 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 14:59:57.148788 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 14:59:57.148788 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 23 14:59:57.148788 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 23 14:59:57.148788 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 23 14:59:57.148788 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 23 14:59:57.150585 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 14:59:57.150585 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 14:59:57.150585 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 23 14:59:57.150585 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 14:59:57.150585 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 23 14:59:57.150585 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 14:59:57.150585 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 14:59:57.150585 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 14:59:57.150585 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 14:59:57.150585 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 14:59:57.150585 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 14:59:57.150585 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Sep 23 14:59:57.150585 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 14:59:57.150585 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Sep 23 14:59:57.150585 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 14:59:57.150585 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 14:59:57.150585 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 14:59:57.150585 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Sep 23 14:59:57.151557 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 14:59:57.151557 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Sep 23 14:59:57.151557 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 23 14:59:57.151557 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 23 14:59:57.151557 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 23 14:59:57.151557 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 23 14:59:57.151557 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 23 14:59:57.151557 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 23 14:59:57.151557 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 23 14:59:57.151557 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Sep 23 14:59:57.151557 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 460/891] 10.222.0.55 () {56 vars in 1117 bytes} [Fri Sep 23 14:59:56 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 14:59:57.156997 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdcf2ae-e86c-4469-9c1c-04030e8d6ef8 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:57.177340 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d7968c-814e-463e-b540-76a14ce521f2 None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:57.177685 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d7968c-814e-463e-b540-76a14ce521f2 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:57.177898 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d7968c-814e-463e-b540-76a14ce521f2 None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:57.179526 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 461/892] 10.222.0.55 () {54 vars in 927 bytes} [Fri Sep 23 14:59:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 23 14:59:57.184846 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43873bc-dcbe-4ddc-a212-5205fcda6fde None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:57.185148 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43873bc-dcbe-4ddc-a212-5205fcda6fde None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:57.185412 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43873bc-dcbe-4ddc-a212-5205fcda6fde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:57.205174 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-f43873bc-dcbe-4ddc-a212-5205fcda6fde None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 14:59:57.206875 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-f43873bc-dcbe-4ddc-a212-5205fcda6fde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 14:59:57.285166 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdcf2ae-e86c-4469-9c1c-04030e8d6ef8 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:57.288621 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 462/893] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Sep 23 14:59:57 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 23 14:59:57.325891 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a669221-1320-4dd3-b57c-cab8a10c8474 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:57.326907 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a669221-1320-4dd3-b57c-cab8a10c8474 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:57.327380 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a669221-1320-4dd3-b57c-cab8a10c8474 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:57.327852 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a669221-1320-4dd3-b57c-cab8a10c8474 None admin] PATH_INFO: `/v3/users` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:57.328835 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a669221-1320-4dd3-b57c-cab8a10c8474 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:59:57.329338 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a669221-1320-4dd3-b57c-cab8a10c8474 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:57.330975 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a669221-1320-4dd3-b57c-cab8a10c8474 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:57.350516 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 432/894] 10.222.0.55 () {56 vars in 1149 bytes} [Fri Sep 23 14:59:57 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:59:57.360211 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43873bc-dcbe-4ddc-a212-5205fcda6fde None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:57.403101 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:57.463160 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43873bc-dcbe-4ddc-a212-5205fcda6fde None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:57.498274 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e076cc6-0b53-4aaa-b554-b96a0a908fc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:57.498886 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e076cc6-0b53-4aaa-b554-b96a0a908fc3 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:57.499168 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e076cc6-0b53-4aaa-b554-b96a0a908fc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:57.499443 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e076cc6-0b53-4aaa-b554-b96a0a908fc3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:57.567247 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a669221-1320-4dd3-b57c-cab8a10c8474 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:57.570647 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e076cc6-0b53-4aaa-b554-b96a0a908fc3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:57.572361 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e076cc6-0b53-4aaa-b554-b96a0a908fc3 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:57.620647 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:57.628420 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:57.682770 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a669221-1320-4dd3-b57c-cab8a10c8474 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:57.692218 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 463/895] 10.222.0.55 () {58 vars in 1282 bytes} [Fri Sep 23 14:59:57 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 23 14:59:57.697618 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e076cc6-0b53-4aaa-b554-b96a0a908fc3 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:57.721142 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd082e5-5598-418e-8ba2-e27af10d806a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:57.721786 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd082e5-5598-418e-8ba2-e27af10d806a None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:57.722061 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd082e5-5598-418e-8ba2-e27af10d806a None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:57.722329 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd082e5-5598-418e-8ba2-e27af10d806a None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:57.722906 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd082e5-5598-418e-8ba2-e27af10d806a None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:57.724402 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd082e5-5598-418e-8ba2-e27af10d806a None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:57.734860 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application [None req-1bd082e5-5598-418e-8ba2-e27af10d806a None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 23 14:59:57.734860 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 14:59:57.734860 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 23 14:59:57.734860 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 23 14:59:57.734860 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 23 14:59:57.734860 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 23 14:59:57.734860 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 23 14:59:57.734860 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 23 14:59:57.734860 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 23 14:59:57.734860 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Sep 23 14:59:57.734860 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 23 14:59:57.734860 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Sep 23 14:59:57.734860 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 14:59:57.734860 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 14:59:57.734860 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 14:59:57.734860 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 14:59:57.735887 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 14:59:57.735887 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 14:59:57.735887 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 14:59:57.735887 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 14:59:57.735887 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 14:59:57.735887 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 14:59:57.735887 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 14:59:57.735887 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 23 14:59:57.735887 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 23 14:59:57.735887 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 23 14:59:57.735887 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 23 14:59:57.735887 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 14:59:57.735887 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 14:59:57.735887 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 14:59:57.735887 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 14:59:57.735887 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 14:59:57.735887 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 14:59:57.736887 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 14:59:57.736887 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application with Lock( Sep 23 14:59:57.736887 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 14:59:57.736887 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._enter() Sep 23 14:59:57.736887 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 14:59:57.736887 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 14:59:57.736887 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 14:59:57.736887 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.creator() Sep 23 14:59:57.736887 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 14:59:57.736887 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application created_value = creator( Sep 23 14:59:57.736887 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 23 14:59:57.736887 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 23 14:59:57.736887 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 23 14:59:57.736887 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Sep 23 14:59:57.740075 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 433/896] 10.222.0.55 () {56 vars in 1132 bytes} [Fri Sep 23 14:59:57 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 14:59:57.741974 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13b8b42-2637-4a51-bc1c-2cf1e9cd091c None None] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:57.742263 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13b8b42-2637-4a51-bc1c-2cf1e9cd091c None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:57.742532 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13b8b42-2637-4a51-bc1c-2cf1e9cd091c None None] PATH_INFO: `/` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:57.744046 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 434/897] 10.222.0.55 () {54 vars in 927 bytes} [Fri Sep 23 14:59:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 14:59:57.748965 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13b8b42-2637-4a51-bc1c-2cf1e9cd091c None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:57.817745 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e076cc6-0b53-4aaa-b554-b96a0a908fc3 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:57.856872 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cce6690-fe52-43e8-b0fc-ffea18e702a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:57.857741 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cce6690-fe52-43e8-b0fc-ffea18e702a7 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:57.858017 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cce6690-fe52-43e8-b0fc-ffea18e702a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:57.858290 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cce6690-fe52-43e8-b0fc-ffea18e702a7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:57.866145 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13b8b42-2637-4a51-bc1c-2cf1e9cd091c None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:57.868934 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cce6690-fe52-43e8-b0fc-ffea18e702a7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:57.870098 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cce6690-fe52-43e8-b0fc-ffea18e702a7 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:57.871905 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:57.885458 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:57.885612 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-4cce6690-fe52-43e8-b0fc-ffea18e702a7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 14:59:57.885612 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 14:59:57.885612 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 14:59:57.885612 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 14:59:57.885612 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 14:59:57.885612 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 14:59:57.885612 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 14:59:57.885612 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 14:59:57.885612 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 14:59:57.885612 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 14:59:57.885612 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 14:59:57.885612 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 14:59:57.885612 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 14:59:57.885612 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 14:59:57.885612 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 14:59:57.885612 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 14:59:57.887483 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 14:59:57.887483 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 14:59:57.887483 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 14:59:57.887483 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 14:59:57.887483 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 14:59:57.887483 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 14:59:57.887483 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 14:59:57.887483 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Sep 23 14:59:57.887483 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 14:59:57.887483 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Sep 23 14:59:57.887483 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 14:59:57.887483 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 14:59:57.887483 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 14:59:57.887483 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Sep 23 14:59:57.887483 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 14:59:57.887483 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Sep 23 14:59:57.887483 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 14:59:57.888810 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 14:59:57.888810 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 14:59:57.888810 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Sep 23 14:59:57.888810 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 464/898] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Sep 23 14:59:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 23 14:59:57.888810 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edf44c8-f312-47f8-8119-428fe21ca367 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:57.888810 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edf44c8-f312-47f8-8119-428fe21ca367 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:57.888810 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edf44c8-f312-47f8-8119-428fe21ca367 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:57.906760 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-6edf44c8-f312-47f8-8119-428fe21ca367 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 14:59:57.908435 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-6edf44c8-f312-47f8-8119-428fe21ca367 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 14:59:57.911760 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c394b2a-e962-45b7-a626-f36e7e74f499 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:57.913232 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c394b2a-e962-45b7-a626-f36e7e74f499 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:57.913517 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c394b2a-e962-45b7-a626-f36e7e74f499 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:57.913794 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c394b2a-e962-45b7-a626-f36e7e74f499 None admin] PATH_INFO: `/v3/services` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:57.914496 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c394b2a-e962-45b7-a626-f36e7e74f499 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:59:57.914797 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c394b2a-e962-45b7-a626-f36e7e74f499 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:57.918929 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c394b2a-e962-45b7-a626-f36e7e74f499 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:57.922729 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 435/899] 10.222.0.55 () {56 vars in 1126 bytes} [Fri Sep 23 14:59:57 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:59:57.923696 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c394b2a-e962-45b7-a626-f36e7e74f499 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:57.989760 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 465/900] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Sep 23 14:59:57 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 23 14:59:57.990427 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edf44c8-f312-47f8-8119-428fe21ca367 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:58.043017 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c394b2a-e962-45b7-a626-f36e7e74f499 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:58.087383 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34309ab2-4b64-40fb-854a-7ac37d1d1b53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:58.088453 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34309ab2-4b64-40fb-854a-7ac37d1d1b53 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:58.089395 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34309ab2-4b64-40fb-854a-7ac37d1d1b53 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:58.089911 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34309ab2-4b64-40fb-854a-7ac37d1d1b53 None admin] PATH_INFO: `/v3/domains` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:58.091166 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34309ab2-4b64-40fb-854a-7ac37d1d1b53 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:59:58.091778 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34309ab2-4b64-40fb-854a-7ac37d1d1b53 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:58.093351 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34309ab2-4b64-40fb-854a-7ac37d1d1b53 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:58.107966 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 436/901] 10.222.0.55 () {56 vars in 1109 bytes} [Fri Sep 23 14:59:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:59:58.113374 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34309ab2-4b64-40fb-854a-7ac37d1d1b53 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:58.126895 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edf44c8-f312-47f8-8119-428fe21ca367 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:58.156827 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:58.159656 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:58.171796 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5487f0-326b-46d2-a0de-6f576d7bcaef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:58.172440 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5487f0-326b-46d2-a0de-6f576d7bcaef None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:58.172858 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5487f0-326b-46d2-a0de-6f576d7bcaef None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:58.173124 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5487f0-326b-46d2-a0de-6f576d7bcaef None admin] PATH_INFO: `/v3/services` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:58.173755 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5487f0-326b-46d2-a0de-6f576d7bcaef None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:59:58.174055 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5487f0-326b-46d2-a0de-6f576d7bcaef None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:58.176256 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5487f0-326b-46d2-a0de-6f576d7bcaef None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:58.187082 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 466/902] 10.222.0.55 () {56 vars in 1126 bytes} [Fri Sep 23 14:59:57 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:59:58.188192 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5487f0-326b-46d2-a0de-6f576d7bcaef None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:58.227055 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34309ab2-4b64-40fb-854a-7ac37d1d1b53 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:58.265294 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5277009f-852c-4158-8ea5-f68b5cbcaa45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:58.266318 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5277009f-852c-4158-8ea5-f68b5cbcaa45 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:58.266867 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5277009f-852c-4158-8ea5-f68b5cbcaa45 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:58.267333 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5277009f-852c-4158-8ea5-f68b5cbcaa45 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:58.274878 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5277009f-852c-4158-8ea5-f68b5cbcaa45 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:58.276606 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5277009f-852c-4158-8ea5-f68b5cbcaa45 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:58.288530 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application [None req-5277009f-852c-4158-8ea5-f68b5cbcaa45 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 14:59:58.288530 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 14:59:58.288530 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 14:59:58.288530 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 14:59:58.288530 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 14:59:58.288530 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 14:59:58.288530 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 14:59:58.288530 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 14:59:58.288530 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 14:59:58.288530 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 14:59:58.288530 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 14:59:58.288530 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 14:59:58.288530 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 23 14:59:58.288530 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 23 14:59:58.288530 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 23 14:59:58.289429 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 23 14:59:58.289429 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 14:59:58.289429 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 14:59:58.289429 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 14:59:58.289429 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 14:59:58.289429 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 14:59:58.289429 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 14:59:58.289429 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 14:59:58.289429 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application with Lock( Sep 23 14:59:58.289429 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 14:59:58.289429 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._enter() Sep 23 14:59:58.289429 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 14:59:58.289429 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 14:59:58.289429 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 14:59:58.289429 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.creator() Sep 23 14:59:58.289429 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 14:59:58.289429 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application created_value = creator( Sep 23 14:59:58.290565 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 23 14:59:58.290565 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 23 14:59:58.290565 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 23 14:59:58.290565 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 23 14:59:58.290565 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 23 14:59:58.290565 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 23 14:59:58.290565 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 14:59:58.290565 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Sep 23 14:59:58.292897 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 437/903] 10.222.0.55 () {56 vars in 1111 bytes} [Fri Sep 23 14:59:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 14:59:58.298363 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5277009f-852c-4158-8ea5-f68b5cbcaa45 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:58.301892 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5487f0-326b-46d2-a0de-6f576d7bcaef None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:58.341042 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce59094c-608a-4a24-b3b4-3a00548ed166 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:58.341691 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce59094c-608a-4a24-b3b4-3a00548ed166 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:58.341974 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce59094c-608a-4a24-b3b4-3a00548ed166 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:58.342248 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce59094c-608a-4a24-b3b4-3a00548ed166 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:58.343084 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce59094c-608a-4a24-b3b4-3a00548ed166 None admin] RBAC: Adding query filter params (interface=public, service_id=48f89297a6ee4e33af3945e9c8a0ca0c, region_id=RegionOne) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:59:58.343391 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce59094c-608a-4a24-b3b4-3a00548ed166 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:58.345095 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce59094c-608a-4a24-b3b4-3a00548ed166 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:58.351378 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 467/904] 10.222.0.55 () {56 vars in 1251 bytes} [Fri Sep 23 14:59:58 2022] GET /identity/v3/endpoints?service_id=48f89297a6ee4e33af3945e9c8a0ca0c&interface=public®ion_id=RegionOne => generated 194 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 14:59:58.352251 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce59094c-608a-4a24-b3b4-3a00548ed166 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:58.405559 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:58.409158 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5277009f-852c-4158-8ea5-f68b5cbcaa45 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:58.412726 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:58.442848 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab541688-8513-4eb1-b6d3-baa571f34e09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:58.444269 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab541688-8513-4eb1-b6d3-baa571f34e09 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:58.445004 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab541688-8513-4eb1-b6d3-baa571f34e09 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:58.445516 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab541688-8513-4eb1-b6d3-baa571f34e09 None admin] PATH_INFO: `/v3/projects` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:58.447327 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab541688-8513-4eb1-b6d3-baa571f34e09 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:59:58.447708 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab541688-8513-4eb1-b6d3-baa571f34e09 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:58.449185 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab541688-8513-4eb1-b6d3-baa571f34e09 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:58.461337 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 438/905] 10.222.0.55 () {56 vars in 1148 bytes} [Fri Sep 23 14:59:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 14:59:58.462315 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab541688-8513-4eb1-b6d3-baa571f34e09 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:58.468640 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce59094c-608a-4a24-b3b4-3a00548ed166 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:58.506454 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad396bb-546a-45e6-b76e-22f6cd378420 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:58.507106 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad396bb-546a-45e6-b76e-22f6cd378420 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:58.507390 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad396bb-546a-45e6-b76e-22f6cd378420 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:58.507669 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad396bb-546a-45e6-b76e-22f6cd378420 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:58.508707 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad396bb-546a-45e6-b76e-22f6cd378420 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:58.511562 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad396bb-546a-45e6-b76e-22f6cd378420 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:58.520876 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-8ad396bb-546a-45e6-b76e-22f6cd378420 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 23 14:59:58.520876 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 14:59:58.520876 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 23 14:59:58.520876 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 23 14:59:58.520876 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 23 14:59:58.520876 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 23 14:59:58.520876 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 23 14:59:58.520876 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 23 14:59:58.520876 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 23 14:59:58.520876 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Sep 23 14:59:58.520876 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 23 14:59:58.520876 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Sep 23 14:59:58.520876 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 14:59:58.520876 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 14:59:58.520876 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 14:59:58.520876 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 14:59:58.520876 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 14:59:58.522025 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 14:59:58.522025 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 14:59:58.522025 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 14:59:58.522025 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 14:59:58.522025 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 14:59:58.522025 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 14:59:58.522025 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 23 14:59:58.522025 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 23 14:59:58.522025 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 23 14:59:58.522025 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 23 14:59:58.522025 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 14:59:58.522025 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 14:59:58.522025 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 14:59:58.522025 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 14:59:58.522025 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 14:59:58.522025 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 14:59:58.522933 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 14:59:58.522933 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Sep 23 14:59:58.522933 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 14:59:58.522933 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Sep 23 14:59:58.522933 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 14:59:58.522933 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 14:59:58.522933 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 14:59:58.522933 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Sep 23 14:59:58.522933 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 14:59:58.522933 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Sep 23 14:59:58.522933 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 23 14:59:58.522933 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 23 14:59:58.522933 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 23 14:59:58.522933 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Sep 23 14:59:58.522933 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 468/906] 10.222.0.55 () {56 vars in 1114 bytes} [Fri Sep 23 14:59:58 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 14:59:58.523723 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad396bb-546a-45e6-b76e-22f6cd378420 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:58.575648 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab541688-8513-4eb1-b6d3-baa571f34e09 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:58.611799 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ce1e24-8ec0-4c64-8362-1329cd2f8ca7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:58.612443 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ce1e24-8ec0-4c64-8362-1329cd2f8ca7 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:58.612722 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ce1e24-8ec0-4c64-8362-1329cd2f8ca7 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:58.612995 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ce1e24-8ec0-4c64-8362-1329cd2f8ca7 None admin] PATH_INFO: `/v3/services` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:58.613589 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ce1e24-8ec0-4c64-8362-1329cd2f8ca7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:58.614937 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ce1e24-8ec0-4c64-8362-1329cd2f8ca7 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:58.630655 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 439/907] 10.222.0.55 () {56 vars in 1087 bytes} [Fri Sep 23 14:59:58 2022] GET /identity/v3/services => generated 1497 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 14:59:58.630655 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ce1e24-8ec0-4c64-8362-1329cd2f8ca7 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:58.635587 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad396bb-546a-45e6-b76e-22f6cd378420 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:58.657478 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:58.671718 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:58.686543 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0af779-46d4-4db1-b67c-867351b09f9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:58.687173 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0af779-46d4-4db1-b67c-867351b09f9a None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:58.687451 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0af779-46d4-4db1-b67c-867351b09f9a None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:58.687723 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0af779-46d4-4db1-b67c-867351b09f9a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:58.688511 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0af779-46d4-4db1-b67c-867351b09f9a None admin] RBAC: Adding query filter params (role.id=f2f7d48ece4f42f682922c1f1beefbd9, scope.project.id=87b3e9d4b6474bd4a04ba59ef52bf3e2, user.id=68206a6b4f244d45bf7bf1504396e430) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:59:58.688821 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0af779-46d4-4db1-b67c-867351b09f9a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:58.690981 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0af779-46d4-4db1-b67c-867351b09f9a None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:58.700252 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 469/908] 10.222.0.55 () {56 vars in 1374 bytes} [Fri Sep 23 14:59:58 2022] GET /identity/v3/role_assignments?user.id=68206a6b4f244d45bf7bf1504396e430&scope.project.id=87b3e9d4b6474bd4a04ba59ef52bf3e2&role.id=f2f7d48ece4f42f682922c1f1beefbd9 => generated 259 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:59:58.782199 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ce1e24-8ec0-4c64-8362-1329cd2f8ca7 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:58.827613 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc9a580-81f2-48cf-b5c3-653de7590b59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:58.828322 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc9a580-81f2-48cf-b5c3-653de7590b59 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:58.828605 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc9a580-81f2-48cf-b5c3-653de7590b59 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:58.828885 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc9a580-81f2-48cf-b5c3-653de7590b59 None admin] PATH_INFO: `/v3/services` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:58.829558 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc9a580-81f2-48cf-b5c3-653de7590b59 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:59:58.829866 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc9a580-81f2-48cf-b5c3-653de7590b59 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:58.831372 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc9a580-81f2-48cf-b5c3-653de7590b59 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:58.838535 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 440/909] 10.222.0.55 () {56 vars in 1114 bytes} [Fri Sep 23 14:59:58 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:59:58.844955 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0af779-46d4-4db1-b67c-867351b09f9a None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:58.967931 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0af779-46d4-4db1-b67c-867351b09f9a None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:59.008622 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f80169-4e03-4810-8ca0-7b3514817fe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:59.009345 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f80169-4e03-4810-8ca0-7b3514817fe8 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:59.009631 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f80169-4e03-4810-8ca0-7b3514817fe8 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:59.009906 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f80169-4e03-4810-8ca0-7b3514817fe8 None admin] PATH_INFO: `/v3/services` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:59.010544 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f80169-4e03-4810-8ca0-7b3514817fe8 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:59:59.010837 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f80169-4e03-4810-8ca0-7b3514817fe8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:59.012233 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f80169-4e03-4810-8ca0-7b3514817fe8 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:59.022471 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 470/910] 10.222.0.55 () {56 vars in 1114 bytes} [Fri Sep 23 14:59:58 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:59:59.023893 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc9a580-81f2-48cf-b5c3-653de7590b59 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:59.089464 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1ecad4-9cd2-4505-a709-858043d4258f None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:59.089761 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1ecad4-9cd2-4505-a709-858043d4258f None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:59.090034 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1ecad4-9cd2-4505-a709-858043d4258f None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:59.091375 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 471/911] 10.222.0.55 () {54 vars in 927 bytes} [Fri Sep 23 14:59:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 23 14:59:59.096599 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e11f65-db9b-4c6c-b28c-e85334b96942 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:59.096889 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e11f65-db9b-4c6c-b28c-e85334b96942 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:59.097174 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e11f65-db9b-4c6c-b28c-e85334b96942 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:59.117008 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-09e11f65-db9b-4c6c-b28c-e85334b96942 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 14:59:59.118241 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-09e11f65-db9b-4c6c-b28c-e85334b96942 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 14:59:59.141993 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:59.155631 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc9a580-81f2-48cf-b5c3-653de7590b59 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:59.167078 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:59.200977 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fcf302-1d26-4883-9f83-a6d2a7f4d3b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:59.201602 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fcf302-1d26-4883-9f83-a6d2a7f4d3b7 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:59.201880 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fcf302-1d26-4883-9f83-a6d2a7f4d3b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:59.202154 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fcf302-1d26-4883-9f83-a6d2a7f4d3b7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:59.202862 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fcf302-1d26-4883-9f83-a6d2a7f4d3b7 None admin] RBAC: Adding query filter params (interface=public, service_id=ca35e508c9e4497ebcfc49b64e9c4f16, region_id=RegionOne) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:59:59.203174 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fcf302-1d26-4883-9f83-a6d2a7f4d3b7 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:59.208010 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fcf302-1d26-4883-9f83-a6d2a7f4d3b7 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:59.210501 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 472/912] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Sep 23 14:59:59 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 23 14:59:59.218398 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 441/913] 10.222.0.55 () {56 vars in 1251 bytes} [Fri Sep 23 14:59:59 2022] GET /identity/v3/endpoints?service_id=ca35e508c9e4497ebcfc49b64e9c4f16&interface=public®ion_id=RegionOne => generated 194 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:59:59.219789 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e11f65-db9b-4c6c-b28c-e85334b96942 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:59.331547 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e11f65-db9b-4c6c-b28c-e85334b96942 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:59.369666 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9fd706-c159-4b13-9f80-d9b99e527e53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:59.370317 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9fd706-c159-4b13-9f80-d9b99e527e53 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:59.370584 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9fd706-c159-4b13-9f80-d9b99e527e53 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:59.370914 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9fd706-c159-4b13-9f80-d9b99e527e53 None admin] PATH_INFO: `/v3/services` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:59.371519 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9fd706-c159-4b13-9f80-d9b99e527e53 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:59.372889 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9fd706-c159-4b13-9f80-d9b99e527e53 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:59.379642 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 473/914] 10.222.0.55 () {56 vars in 1087 bytes} [Fri Sep 23 14:59:59 2022] GET /identity/v3/services => generated 1497 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 23 14:59:59.526543 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fcf302-1d26-4883-9f83-a6d2a7f4d3b7 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:59.641509 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fcf302-1d26-4883-9f83-a6d2a7f4d3b7 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:59.679510 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e49557d-24b5-4c9c-861d-45c353f2a9ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:59.680491 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e49557d-24b5-4c9c-861d-45c353f2a9ba None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:59.680973 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e49557d-24b5-4c9c-861d-45c353f2a9ba None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:59.681404 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e49557d-24b5-4c9c-861d-45c353f2a9ba None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:59.682279 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e49557d-24b5-4c9c-861d-45c353f2a9ba None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:59.683860 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e49557d-24b5-4c9c-861d-45c353f2a9ba None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:59.694102 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application [None req-8e49557d-24b5-4c9c-861d-45c353f2a9ba None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 23 14:59:59.694102 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 14:59:59.694102 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 23 14:59:59.694102 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 23 14:59:59.694102 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 23 14:59:59.694102 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 23 14:59:59.694102 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 23 14:59:59.694102 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 23 14:59:59.694102 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 23 14:59:59.694102 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Sep 23 14:59:59.694102 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 23 14:59:59.694102 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Sep 23 14:59:59.694102 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 14:59:59.694102 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 14:59:59.694102 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 14:59:59.694102 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 14:59:59.694102 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 14:59:59.695386 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 14:59:59.695386 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 14:59:59.695386 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 14:59:59.695386 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 14:59:59.695386 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 14:59:59.695386 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 14:59:59.695386 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 23 14:59:59.695386 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 23 14:59:59.695386 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 23 14:59:59.695386 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 23 14:59:59.695386 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 14:59:59.695386 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 14:59:59.695386 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 14:59:59.695386 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 14:59:59.695386 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 14:59:59.695386 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 14:59:59.697135 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 14:59:59.697135 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application with Lock( Sep 23 14:59:59.697135 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 14:59:59.697135 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._enter() Sep 23 14:59:59.697135 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 14:59:59.697135 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 14:59:59.697135 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 14:59:59.697135 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.creator() Sep 23 14:59:59.697135 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 14:59:59.697135 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application created_value = creator( Sep 23 14:59:59.697135 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 23 14:59:59.697135 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 23 14:59:59.697135 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 23 14:59:59.697135 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Sep 23 14:59:59.700043 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 442/915] 10.222.0.55 () {56 vars in 1126 bytes} [Fri Sep 23 14:59:59 2022] GET /identity/v3/services/object-store => generated 93 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 14:59:59.708496 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9fd706-c159-4b13-9f80-d9b99e527e53 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:59.747616 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:59.814954 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9fd706-c159-4b13-9f80-d9b99e527e53 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 14:59:59.855214 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b7ff30-4d34-4c02-9451-44e3f8a162e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 14:59:59.855867 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b7ff30-4d34-4c02-9451-44e3f8a162e2 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 14:59:59.856133 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b7ff30-4d34-4c02-9451-44e3f8a162e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 14:59:59.856392 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b7ff30-4d34-4c02-9451-44e3f8a162e2 None admin] PATH_INFO: `/v3/services` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 14:59:59.857026 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b7ff30-4d34-4c02-9451-44e3f8a162e2 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 14:59:59.857328 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b7ff30-4d34-4c02-9451-44e3f8a162e2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 14:59:59.858680 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b7ff30-4d34-4c02-9451-44e3f8a162e2 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 14:59:59.863941 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 474/916] 10.222.0.55 () {56 vars in 1122 bytes} [Fri Sep 23 14:59:59 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 14:59:59.873314 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e49557d-24b5-4c9c-861d-45c353f2a9ba None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 14:59:59.895168 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 14:59:59.984523 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e49557d-24b5-4c9c-861d-45c353f2a9ba None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:00:00.022274 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f55783-a3d7-42f8-9650-09fc15ea8c30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:00:00.022891 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f55783-a3d7-42f8-9650-09fc15ea8c30 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:00.023171 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f55783-a3d7-42f8-9650-09fc15ea8c30 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:00.023445 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f55783-a3d7-42f8-9650-09fc15ea8c30 None admin] PATH_INFO: `/v3/services` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:00.024103 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f55783-a3d7-42f8-9650-09fc15ea8c30 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:00:00.024407 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f55783-a3d7-42f8-9650-09fc15ea8c30 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:00:00.025917 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f55783-a3d7-42f8-9650-09fc15ea8c30 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:00:00.031487 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 443/917] 10.222.0.55 () {56 vars in 1122 bytes} [Fri Sep 23 14:59:59 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:00:00.037574 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b7ff30-4d34-4c02-9451-44e3f8a162e2 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:00:00.157987 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b7ff30-4d34-4c02-9451-44e3f8a162e2 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:00:00.159463 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:00:00.194499 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34ccf3e-2864-4e40-b837-467df50a93b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:00:00.195162 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34ccf3e-2864-4e40-b837-467df50a93b5 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:00.195443 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34ccf3e-2864-4e40-b837-467df50a93b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:00.195715 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34ccf3e-2864-4e40-b837-467df50a93b5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:00.196726 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34ccf3e-2864-4e40-b837-467df50a93b5 None admin] RBAC: Adding query filter params (interface=public, service_id=c7b748108d7d4bfe9fad4a8d8a9e6fc3, region_id=RegionOne) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:00:00.197058 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34ccf3e-2864-4e40-b837-467df50a93b5 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:00:00.198421 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34ccf3e-2864-4e40-b837-467df50a93b5 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:00:00.204198 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 475/918] 10.222.0.55 () {56 vars in 1251 bytes} [Fri Sep 23 15:00:00 2022] GET /identity/v3/endpoints?service_id=c7b748108d7d4bfe9fad4a8d8a9e6fc3&interface=public®ion_id=RegionOne => generated 194 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:00:00.211088 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f55783-a3d7-42f8-9650-09fc15ea8c30 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:00:00.302937 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:00:00.323641 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f55783-a3d7-42f8-9650-09fc15ea8c30 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:00:00.358541 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca1fc0c-c961-4fc8-8fe0-e3c05de35801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:00:00.359547 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca1fc0c-c961-4fc8-8fe0-e3c05de35801 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:00.360073 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca1fc0c-c961-4fc8-8fe0-e3c05de35801 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:00.360736 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca1fc0c-c961-4fc8-8fe0-e3c05de35801 None admin] PATH_INFO: `/v3/services` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:00.361556 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca1fc0c-c961-4fc8-8fe0-e3c05de35801 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:00:00.363118 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca1fc0c-c961-4fc8-8fe0-e3c05de35801 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:00:00.372104 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 444/919] 10.222.0.55 () {56 vars in 1087 bytes} [Fri Sep 23 15:00:00 2022] GET /identity/v3/services => generated 1497 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:00:00.921622 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ab0ce1-404d-4df0-9e90-f49231a50419 None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:00.922193 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ab0ce1-404d-4df0-9e90-f49231a50419 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:00.922193 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ab0ce1-404d-4df0-9e90-f49231a50419 None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:00.923714 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 476/920] 10.222.0.55 () {54 vars in 927 bytes} [Fri Sep 23 15:00:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:00:00.929156 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b64738-e21e-4648-906b-57076a8fb793 None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:00.929586 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b64738-e21e-4648-906b-57076a8fb793 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:00.929758 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b64738-e21e-4648-906b-57076a8fb793 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:00.948745 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-c5b64738-e21e-4648-906b-57076a8fb793 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:00:00.949943 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-c5b64738-e21e-4648-906b-57076a8fb793 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:00:01.013351 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f07625f-d7bb-4ad8-859a-3a908fd2a535 None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:01.013646 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f07625f-d7bb-4ad8-859a-3a908fd2a535 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:01.013919 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f07625f-d7bb-4ad8-859a-3a908fd2a535 None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:01.015530 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 477/921] 10.222.0.55 () {54 vars in 927 bytes} [Fri Sep 23 15:00:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 23 15:00:01.020449 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530dadfc-62f6-4664-a2c2-0624c02a798b None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:01.020737 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530dadfc-62f6-4664-a2c2-0624c02a798b None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:01.021020 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530dadfc-62f6-4664-a2c2-0624c02a798b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:01.021648 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 445/922] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Sep 23 15:00:00 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 23 15:00:01.044357 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-530dadfc-62f6-4664-a2c2-0624c02a798b None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:00:01.045568 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-530dadfc-62f6-4664-a2c2-0624c02a798b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:00:01.128072 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 478/923] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Sep 23 15:00:01 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 23 15:00:01.295949 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b64738-e21e-4648-906b-57076a8fb793 None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:00:01.375865 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:00:01.401820 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b64738-e21e-4648-906b-57076a8fb793 None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:00:01.417810 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530dadfc-62f6-4664-a2c2-0624c02a798b None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:00:01.438926 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b70f737-2a04-43cd-8e6d-d4af5751490f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:00:01.439607 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b70f737-2a04-43cd-8e6d-d4af5751490f None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:01.439869 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b70f737-2a04-43cd-8e6d-d4af5751490f None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:01.440210 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b70f737-2a04-43cd-8e6d-d4af5751490f None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:01.440843 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b70f737-2a04-43cd-8e6d-d4af5751490f None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:00:01.442267 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b70f737-2a04-43cd-8e6d-d4af5751490f None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:00:01.452244 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application [None req-6b70f737-2a04-43cd-8e6d-d4af5751490f None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 23 15:00:01.452244 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:00:01.452244 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 23 15:00:01.452244 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 23 15:00:01.452244 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 23 15:00:01.452244 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 23 15:00:01.452244 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 23 15:00:01.452244 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 23 15:00:01.452244 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 23 15:00:01.452244 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Sep 23 15:00:01.452244 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 23 15:00:01.452244 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Sep 23 15:00:01.452244 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:00:01.452244 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:00:01.452244 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:00:01.452244 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:00:01.453260 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:00:01.453260 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:00:01.453260 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:00:01.453260 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:00:01.453260 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:00:01.453260 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:00:01.453260 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:00:01.453260 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 23 15:00:01.453260 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 23 15:00:01.453260 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 23 15:00:01.453260 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 23 15:00:01.453260 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:00:01.453260 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:00:01.453260 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:00:01.453260 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:00:01.453260 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:00:01.453260 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:00:01.454162 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:00:01.454162 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application with Lock( Sep 23 15:00:01.454162 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:00:01.454162 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:00:01.454162 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:00:01.454162 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:00:01.454162 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:00:01.454162 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:00:01.454162 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:00:01.454162 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:00:01.454162 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 23 15:00:01.454162 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 23 15:00:01.454162 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 23 15:00:01.454162 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Sep 23 15:00:01.454162 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 446/924] 10.222.0.55 () {56 vars in 1132 bytes} [Fri Sep 23 15:00:01 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:00:01.459767 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b70f737-2a04-43cd-8e6d-d4af5751490f None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:00:01.532124 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:00:01.543549 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530dadfc-62f6-4664-a2c2-0624c02a798b None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:00:01.581263 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02250b9-2f51-4e07-a120-2a767365e5c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:00:01.581971 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02250b9-2f51-4e07-a120-2a767365e5c2 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:01.582362 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02250b9-2f51-4e07-a120-2a767365e5c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:01.583016 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02250b9-2f51-4e07-a120-2a767365e5c2 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:01.594745 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02250b9-2f51-4e07-a120-2a767365e5c2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 15:00:01.599892 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-c02250b9-2f51-4e07-a120-2a767365e5c2 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 23 15:00:01.599892 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:00:01.599892 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:00:01.599892 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:00:01.599892 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:00:01.599892 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:00:01.599892 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:00:01.599892 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:00:01.599892 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:00:01.599892 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:00:01.599892 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:00:01.599892 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:00:01.599892 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 15:00:01.599892 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 15:00:01.599892 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 15:00:01.599892 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 15:00:01.600830 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 15:00:01.600830 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 15:00:01.600830 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 15:00:01.600830 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 15:00:01.600830 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:00:01.600830 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:00:01.600830 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:00:01.600830 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:00:01.600830 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:00:01.600830 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:00:01.600830 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:00:01.600830 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Sep 23 15:00:01.600830 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:00:01.600830 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:00:01.600830 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:00:01.600830 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:00:01.601671 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:00:01.601671 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:00:01.601671 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:00:01.601671 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:00:01.601671 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 15:00:01.601671 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 15:00:01.601671 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 15:00:01.601671 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 15:00:01.601671 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 15:00:01.601671 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 15:00:01.601671 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 23 15:00:01.601671 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Sep 23 15:00:01.601671 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 479/925] 10.222.0.55 () {56 vars in 1120 bytes} [Fri Sep 23 15:00:01 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 23 15:00:01.609618 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02250b9-2f51-4e07-a120-2a767365e5c2 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:00:01.612589 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b70f737-2a04-43cd-8e6d-d4af5751490f None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:00:01.653888 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9f34b9-821d-484f-96d3-be7a76758442 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:00:01.654698 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9f34b9-821d-484f-96d3-be7a76758442 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:01.655079 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9f34b9-821d-484f-96d3-be7a76758442 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:01.655449 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9f34b9-821d-484f-96d3-be7a76758442 None admin] PATH_INFO: `/v3/services` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:01.656237 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9f34b9-821d-484f-96d3-be7a76758442 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:00:01.656643 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9f34b9-821d-484f-96d3-be7a76758442 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:00:01.658669 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9f34b9-821d-484f-96d3-be7a76758442 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:00:01.670749 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 447/926] 10.222.0.55 () {56 vars in 1126 bytes} [Fri Sep 23 15:00:01 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 15:00:01.671895 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9f34b9-821d-484f-96d3-be7a76758442 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:00:01.675039 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:00:01.742634 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02250b9-2f51-4e07-a120-2a767365e5c2 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:00:01.791765 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd2fa98-0c58-4ced-9a06-5e6edac53959 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:00:01.792660 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd2fa98-0c58-4ced-9a06-5e6edac53959 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:01.793107 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd2fa98-0c58-4ced-9a06-5e6edac53959 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:01.793540 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd2fa98-0c58-4ced-9a06-5e6edac53959 None admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:01.794381 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd2fa98-0c58-4ced-9a06-5e6edac53959 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:00:01.794825 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd2fa98-0c58-4ced-9a06-5e6edac53959 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:00:01.799828 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd2fa98-0c58-4ced-9a06-5e6edac53959 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:00:01.814934 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 480/927] 10.222.0.55 () {56 vars in 1115 bytes} [Fri Sep 23 15:00:01 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:00:01.820048 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9f34b9-821d-484f-96d3-be7a76758442 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:00:01.821995 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd2fa98-0c58-4ced-9a06-5e6edac53959 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:00:01.843213 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:00:01.860492 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b105f4c6-563e-4ffa-b3bc-f739e5a34283 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:00:01.861495 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b105f4c6-563e-4ffa-b3bc-f739e5a34283 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:01.862006 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b105f4c6-563e-4ffa-b3bc-f739e5a34283 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:01.862335 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b105f4c6-563e-4ffa-b3bc-f739e5a34283 None admin] PATH_INFO: `/v3/services` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:01.863032 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b105f4c6-563e-4ffa-b3bc-f739e5a34283 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:00:01.863335 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b105f4c6-563e-4ffa-b3bc-f739e5a34283 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:00:01.864713 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b105f4c6-563e-4ffa-b3bc-f739e5a34283 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:00:01.870747 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 448/928] 10.222.0.55 () {56 vars in 1126 bytes} [Fri Sep 23 15:00:01 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:00:01.871999 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d452422-943d-453b-ad71-8f7746993fba None None] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:01.872303 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d452422-943d-453b-ad71-8f7746993fba None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:01.872569 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d452422-943d-453b-ad71-8f7746993fba None None] PATH_INFO: `/` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:01.876144 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 449/929] 10.222.0.55 () {54 vars in 927 bytes} [Fri Sep 23 15:00:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:00:01.876144 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d452422-943d-453b-ad71-8f7746993fba None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:00:01.940208 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd2fa98-0c58-4ced-9a06-5e6edac53959 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:00:01.957164 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:00:01.974986 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6202626c-d100-4049-83a9-c953339903f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:00:01.975625 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6202626c-d100-4049-83a9-c953339903f9 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:01.975908 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6202626c-d100-4049-83a9-c953339903f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:01.976215 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6202626c-d100-4049-83a9-c953339903f9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:01.987517 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6202626c-d100-4049-83a9-c953339903f9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:00:01.988973 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6202626c-d100-4049-83a9-c953339903f9 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:00:01.998259 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d452422-943d-453b-ad71-8f7746993fba None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:00:02.004158 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-6202626c-d100-4049-83a9-c953339903f9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 15:00:02.004158 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:00:02.004158 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:00:02.004158 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:00:02.004158 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:00:02.004158 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:00:02.004158 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:00:02.004158 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:00:02.004158 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:00:02.004158 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:00:02.004158 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:00:02.004158 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:00:02.004158 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 15:00:02.004158 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 15:00:02.004158 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 15:00:02.004158 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 15:00:02.005139 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:00:02.005139 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:00:02.005139 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:00:02.005139 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:00:02.005139 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:00:02.005139 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:00:02.005139 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:00:02.005139 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Sep 23 15:00:02.005139 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:00:02.005139 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:00:02.005139 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:00:02.005139 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:00:02.005139 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:00:02.005139 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:00:02.005139 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:00:02.005139 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:00:02.005139 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 15:00:02.006058 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 15:00:02.006058 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 15:00:02.006058 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Sep 23 15:00:02.006058 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 481/930] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Sep 23 15:00:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 23 15:00:02.007091 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd332a15-3413-4749-8a46-31fe66126d83 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:02.007091 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd332a15-3413-4749-8a46-31fe66126d83 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:02.007091 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd332a15-3413-4749-8a46-31fe66126d83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:02.028911 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-dd332a15-3413-4749-8a46-31fe66126d83 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:00:02.030116 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-dd332a15-3413-4749-8a46-31fe66126d83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:00:02.038813 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057a2d88-9d64-4be2-b21d-3795668e69f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:00:02.039188 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057a2d88-9d64-4be2-b21d-3795668e69f6 None admin] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:02.039468 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057a2d88-9d64-4be2-b21d-3795668e69f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:02.039740 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057a2d88-9d64-4be2-b21d-3795668e69f6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:02.040779 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057a2d88-9d64-4be2-b21d-3795668e69f6 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:00:02.042165 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057a2d88-9d64-4be2-b21d-3795668e69f6 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:00:02.096476 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:00:02.100932 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 482/931] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Sep 23 15:00:02 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 23 15:00:02.101740 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd332a15-3413-4749-8a46-31fe66126d83 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:00:02.153045 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 450/932] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Sep 23 15:00:01 2022] POST /identity/v3/endpoints => generated 347 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:00:02.220618 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd332a15-3413-4749-8a46-31fe66126d83 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:00:02.267243 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d154abc9-a47c-4a20-9059-45973e663106 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:00:02.268116 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d154abc9-a47c-4a20-9059-45973e663106 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:02.268404 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d154abc9-a47c-4a20-9059-45973e663106 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:02.268681 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d154abc9-a47c-4a20-9059-45973e663106 None admin] PATH_INFO: `/v3/domains` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:02.269348 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d154abc9-a47c-4a20-9059-45973e663106 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:00:02.269653 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d154abc9-a47c-4a20-9059-45973e663106 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:00:02.280027 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d154abc9-a47c-4a20-9059-45973e663106 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:00:02.294732 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 483/933] 10.222.0.55 () {56 vars in 1109 bytes} [Fri Sep 23 15:00:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:00:02.303307 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057a2d88-9d64-4be2-b21d-3795668e69f6 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:00:02.402250 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d154abc9-a47c-4a20-9059-45973e663106 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:00:02.419150 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057a2d88-9d64-4be2-b21d-3795668e69f6 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:00:02.443535 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:00:02.455317 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c98c6b-32f5-4288-a377-8ac423cef172 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:00:02.455969 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c98c6b-32f5-4288-a377-8ac423cef172 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:02.456273 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c98c6b-32f5-4288-a377-8ac423cef172 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:02.456544 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c98c6b-32f5-4288-a377-8ac423cef172 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:02.465012 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c98c6b-32f5-4288-a377-8ac423cef172 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:00:02.466676 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c98c6b-32f5-4288-a377-8ac423cef172 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:00:02.476043 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:00:02.479415 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application [None req-14c98c6b-32f5-4288-a377-8ac423cef172 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 23 15:00:02.479415 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:00:02.479415 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:00:02.479415 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:00:02.479415 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:00:02.479415 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:00:02.479415 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:00:02.479415 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:00:02.479415 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:00:02.479415 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:00:02.479415 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:00:02.479415 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:00:02.479415 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 23 15:00:02.479415 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 23 15:00:02.479415 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 23 15:00:02.479415 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 23 15:00:02.480400 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:00:02.480400 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:00:02.480400 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 23 15:00:02.480400 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 15:00:02.480400 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 23 15:00:02.480400 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 15:00:02.480400 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:00:02.480400 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:00:02.480400 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:00:02.480400 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:00:02.480400 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:00:02.480400 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application with Lock( Sep 23 15:00:02.480400 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:00:02.480400 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:00:02.480400 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:00:02.480400 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:00:02.480400 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:00:02.480400 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:00:02.481357 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:00:02.481357 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:00:02.481357 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 23 15:00:02.481357 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 23 15:00:02.481357 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 23 15:00:02.481357 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 23 15:00:02.481357 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 23 15:00:02.481357 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 23 15:00:02.481357 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 23 15:00:02.481357 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Sep 23 15:00:02.481357 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 451/934] 10.222.0.55 () {56 vars in 1117 bytes} [Fri Sep 23 15:00:02 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:00:02.486697 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c98c6b-32f5-4288-a377-8ac423cef172 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:00:02.527958 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d154abc9-a47c-4a20-9059-45973e663106 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:00:02.568532 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1454769a-24cb-41df-814e-6cf6d3320f09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:00:02.569653 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1454769a-24cb-41df-814e-6cf6d3320f09 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:02.570170 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1454769a-24cb-41df-814e-6cf6d3320f09 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:02.570753 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1454769a-24cb-41df-814e-6cf6d3320f09 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:02.571711 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1454769a-24cb-41df-814e-6cf6d3320f09 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:00:02.573586 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1454769a-24cb-41df-814e-6cf6d3320f09 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:00:02.584658 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-1454769a-24cb-41df-814e-6cf6d3320f09 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 23 15:00:02.584658 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:00:02.584658 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 23 15:00:02.584658 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 23 15:00:02.584658 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 23 15:00:02.584658 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 23 15:00:02.584658 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 23 15:00:02.584658 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 23 15:00:02.584658 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 23 15:00:02.584658 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Sep 23 15:00:02.584658 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 23 15:00:02.584658 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Sep 23 15:00:02.584658 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:00:02.584658 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:00:02.584658 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:00:02.584658 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:00:02.584658 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:00:02.585750 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:00:02.585750 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:00:02.585750 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:00:02.585750 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:00:02.585750 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:00:02.585750 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:00:02.585750 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 23 15:00:02.585750 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 23 15:00:02.585750 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 23 15:00:02.585750 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 23 15:00:02.585750 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:00:02.585750 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:00:02.585750 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:00:02.585750 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:00:02.585750 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:00:02.585750 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:00:02.586616 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:00:02.586616 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Sep 23 15:00:02.586616 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:00:02.586616 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:00:02.586616 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:00:02.586616 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:00:02.586616 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:00:02.586616 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:00:02.586616 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:00:02.586616 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:00:02.586616 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 23 15:00:02.586616 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 23 15:00:02.586616 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 23 15:00:02.586616 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Sep 23 15:00:02.589767 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 484/935] 10.222.0.55 () {56 vars in 1114 bytes} [Fri Sep 23 15:00:02 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:00:02.595841 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1454769a-24cb-41df-814e-6cf6d3320f09 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:00:02.630171 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c98c6b-32f5-4288-a377-8ac423cef172 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:00:02.674564 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b6b469-222d-4bbb-8eb2-90b30897c01a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:00:02.675104 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b6b469-222d-4bbb-8eb2-90b30897c01a None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:02.675428 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b6b469-222d-4bbb-8eb2-90b30897c01a None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:02.676536 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b6b469-222d-4bbb-8eb2-90b30897c01a None admin] PATH_INFO: `/v3/users` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:02.677093 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b6b469-222d-4bbb-8eb2-90b30897c01a None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:00:02.677399 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b6b469-222d-4bbb-8eb2-90b30897c01a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:00:02.679925 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b6b469-222d-4bbb-8eb2-90b30897c01a None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:00:02.702299 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 452/936] 10.222.0.55 () {56 vars in 1149 bytes} [Fri Sep 23 15:00:02 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:00:02.703653 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f77b5ab-4dcb-4ea9-8656-d3fc6bc0961b None None] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:02.703939 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f77b5ab-4dcb-4ea9-8656-d3fc6bc0961b None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:02.704269 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f77b5ab-4dcb-4ea9-8656-d3fc6bc0961b None None] PATH_INFO: `/` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:02.705720 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 453/937] 10.222.0.55 () {54 vars in 927 bytes} [Fri Sep 23 15:00:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:00:02.709791 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f77b5ab-4dcb-4ea9-8656-d3fc6bc0961b None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:00:02.721760 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1454769a-24cb-41df-814e-6cf6d3320f09 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:00:02.748630 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:00:02.766502 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:00:02.766770 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a99a65-695f-4203-b16e-4e314e92a54e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:00:02.767312 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a99a65-695f-4203-b16e-4e314e92a54e None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:02.767594 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a99a65-695f-4203-b16e-4e314e92a54e None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:02.767875 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a99a65-695f-4203-b16e-4e314e92a54e None admin] PATH_INFO: `/v3/services` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:02.768542 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a99a65-695f-4203-b16e-4e314e92a54e None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:00:02.768851 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a99a65-695f-4203-b16e-4e314e92a54e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:00:02.770298 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a99a65-695f-4203-b16e-4e314e92a54e None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:00:02.776219 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 485/938] 10.222.0.55 () {56 vars in 1114 bytes} [Fri Sep 23 15:00:02 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 15:00:02.777247 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b055bc-4e88-476b-82b9-c1df2af7d057 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:02.777608 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b055bc-4e88-476b-82b9-c1df2af7d057 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:02.777860 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b055bc-4e88-476b-82b9-c1df2af7d057 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:02.796620 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-c3b055bc-4e88-476b-82b9-c1df2af7d057 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:00:02.798004 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-c3b055bc-4e88-476b-82b9-c1df2af7d057 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:00:02.845229 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f77b5ab-4dcb-4ea9-8656-d3fc6bc0961b None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:00:02.887041 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd470b1-50e8-4237-a8d4-5b7ebf1e6ec6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:00:02.887685 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd470b1-50e8-4237-a8d4-5b7ebf1e6ec6 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:02.887965 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd470b1-50e8-4237-a8d4-5b7ebf1e6ec6 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:02.888201 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 486/939] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Sep 23 15:00:02 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 23 15:00:02.888334 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd470b1-50e8-4237-a8d4-5b7ebf1e6ec6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:02.889655 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b055bc-4e88-476b-82b9-c1df2af7d057 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:00:02.903669 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd470b1-50e8-4237-a8d4-5b7ebf1e6ec6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:00:02.905994 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd470b1-50e8-4237-a8d4-5b7ebf1e6ec6 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:00:02.925957 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application [None req-ccd470b1-50e8-4237-a8d4-5b7ebf1e6ec6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 15:00:02.925957 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:00:02.925957 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:00:02.925957 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:00:02.925957 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:00:02.925957 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:00:02.925957 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:00:02.925957 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:00:02.925957 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:00:02.925957 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:00:02.925957 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:00:02.925957 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:00:02.925957 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 15:00:02.925957 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 15:00:02.925957 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 15:00:02.925957 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 15:00:02.927031 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:00:02.927031 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:00:02.927031 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:00:02.927031 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:00:02.927031 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:00:02.927031 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:00:02.927031 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:00:02.927031 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application with Lock( Sep 23 15:00:02.927031 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:00:02.927031 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:00:02.927031 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:00:02.927031 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:00:02.927031 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:00:02.927031 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:00:02.927031 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:00:02.927031 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:00:02.927031 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 15:00:02.928106 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 15:00:02.928106 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 15:00:02.928106 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Sep 23 15:00:02.930097 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 454/940] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Sep 23 15:00:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:00:02.936386 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd470b1-50e8-4237-a8d4-5b7ebf1e6ec6 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:00:03.035583 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b055bc-4e88-476b-82b9-c1df2af7d057 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:00:03.053195 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:00:03.066107 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:00:03.079129 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd470b1-50e8-4237-a8d4-5b7ebf1e6ec6 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:00:03.083997 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779bc965-71bb-499e-9adb-928de3960dfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:00:03.084869 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779bc965-71bb-499e-9adb-928de3960dfd None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:03.085623 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779bc965-71bb-499e-9adb-928de3960dfd None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:03.086107 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779bc965-71bb-499e-9adb-928de3960dfd None admin] PATH_INFO: `/v3/services` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:03.086934 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779bc965-71bb-499e-9adb-928de3960dfd None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:00:03.088128 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779bc965-71bb-499e-9adb-928de3960dfd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:00:03.089737 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779bc965-71bb-499e-9adb-928de3960dfd None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:00:03.095596 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 487/941] 10.222.0.55 () {56 vars in 1114 bytes} [Fri Sep 23 15:00:02 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:00:03.109519 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779bc965-71bb-499e-9adb-928de3960dfd None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:00:03.122206 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12d1aeb-5e82-4143-addf-1e807728de30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:00:03.123097 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12d1aeb-5e82-4143-addf-1e807728de30 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:03.126068 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12d1aeb-5e82-4143-addf-1e807728de30 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:03.126218 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12d1aeb-5e82-4143-addf-1e807728de30 None admin] PATH_INFO: `/v3/domains` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:03.126794 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12d1aeb-5e82-4143-addf-1e807728de30 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:00:03.127097 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12d1aeb-5e82-4143-addf-1e807728de30 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:00:03.128517 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12d1aeb-5e82-4143-addf-1e807728de30 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:00:03.143797 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 455/942] 10.222.0.55 () {56 vars in 1109 bytes} [Fri Sep 23 15:00:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:00:03.152032 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12d1aeb-5e82-4143-addf-1e807728de30 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:00:03.245031 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779bc965-71bb-499e-9adb-928de3960dfd None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:00:03.283178 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d8e9d4-2adc-45a0-b066-fb1ca829001a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:00:03.283799 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d8e9d4-2adc-45a0-b066-fb1ca829001a None admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:03.284108 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d8e9d4-2adc-45a0-b066-fb1ca829001a None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:03.284384 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d8e9d4-2adc-45a0-b066-fb1ca829001a None admin] PATH_INFO: `/v3/endpoints` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:03.285055 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d8e9d4-2adc-45a0-b066-fb1ca829001a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:00:03.286422 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d8e9d4-2adc-45a0-b066-fb1ca829001a None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:00:03.293033 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12d1aeb-5e82-4143-addf-1e807728de30 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:00:03.340514 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54183830-cbcc-4316-85f7-02a2700a2eeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:00:03.341162 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54183830-cbcc-4316-85f7-02a2700a2eeb None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:03.341444 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54183830-cbcc-4316-85f7-02a2700a2eeb None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:03.341719 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54183830-cbcc-4316-85f7-02a2700a2eeb None admin] PATH_INFO: `/v3/projects/service` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:03.350466 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54183830-cbcc-4316-85f7-02a2700a2eeb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:00:03.356576 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54183830-cbcc-4316-85f7-02a2700a2eeb None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:00:03.363573 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application [None req-54183830-cbcc-4316-85f7-02a2700a2eeb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 15:00:03.363573 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:00:03.363573 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:00:03.363573 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:00:03.363573 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:00:03.363573 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:00:03.363573 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:00:03.363573 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:00:03.363573 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:00:03.363573 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:00:03.363573 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:00:03.363573 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:00:03.363573 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 23 15:00:03.363573 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 23 15:00:03.363573 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 23 15:00:03.364669 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 23 15:00:03.364669 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:00:03.364669 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:00:03.364669 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:00:03.364669 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:00:03.364669 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:00:03.364669 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:00:03.364669 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:00:03.364669 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application with Lock( Sep 23 15:00:03.364669 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:00:03.364669 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:00:03.364669 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:00:03.364669 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:00:03.364669 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:00:03.364669 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:00:03.364669 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:00:03.364669 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:00:03.365996 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 23 15:00:03.365996 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 23 15:00:03.365996 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 23 15:00:03.365996 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 23 15:00:03.365996 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 23 15:00:03.365996 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 23 15:00:03.365996 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 15:00:03.365996 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Sep 23 15:00:03.368136 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 456/943] 10.222.0.55 () {56 vars in 1111 bytes} [Fri Sep 23 15:00:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 23 15:00:03.369110 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54183830-cbcc-4316-85f7-02a2700a2eeb None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:00:03.385748 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:00:03.391438 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 488/944] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Sep 23 15:00:03 2022] POST /identity/v3/endpoints => generated 346 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:00:03.393437 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d8e9d4-2adc-45a0-b066-fb1ca829001a None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:00:03.462755 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:00:03.516646 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54183830-cbcc-4316-85f7-02a2700a2eeb None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:00:03.564157 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d8e9d4-2adc-45a0-b066-fb1ca829001a None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:00:03.570874 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054d45b4-aec3-47a9-bc4d-29be0e8647b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:00:03.572391 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054d45b4-aec3-47a9-bc4d-29be0e8647b4 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:03.572760 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054d45b4-aec3-47a9-bc4d-29be0e8647b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:03.573154 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054d45b4-aec3-47a9-bc4d-29be0e8647b4 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:03.573942 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054d45b4-aec3-47a9-bc4d-29be0e8647b4 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:00:03.575409 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054d45b4-aec3-47a9-bc4d-29be0e8647b4 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:00:03.586072 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application [None req-054d45b4-aec3-47a9-bc4d-29be0e8647b4 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 23 15:00:03.586072 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:00:03.586072 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 23 15:00:03.586072 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 23 15:00:03.586072 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 23 15:00:03.586072 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 23 15:00:03.586072 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 23 15:00:03.586072 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 23 15:00:03.586072 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 23 15:00:03.586072 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Sep 23 15:00:03.586072 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 23 15:00:03.586072 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Sep 23 15:00:03.586072 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:00:03.586072 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:00:03.586072 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:00:03.586072 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:00:03.586072 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:00:03.587660 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:00:03.587660 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:00:03.587660 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:00:03.587660 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:00:03.587660 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:00:03.587660 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:00:03.587660 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 23 15:00:03.587660 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 23 15:00:03.587660 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 23 15:00:03.587660 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 23 15:00:03.587660 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:00:03.587660 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:00:03.587660 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:00:03.587660 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:00:03.587660 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:00:03.587660 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:00:03.589065 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:00:03.589065 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application with Lock( Sep 23 15:00:03.589065 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:00:03.589065 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:00:03.589065 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:00:03.589065 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:00:03.589065 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:00:03.589065 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:00:03.589065 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:00:03.589065 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:00:03.589065 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 23 15:00:03.589065 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 23 15:00:03.589065 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 23 15:00:03.589065 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Sep 23 15:00:03.589065 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 457/945] 10.222.0.55 () {56 vars in 1126 bytes} [Fri Sep 23 15:00:03 2022] GET /identity/v3/services/object-store => generated 93 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:00:03.595308 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054d45b4-aec3-47a9-bc4d-29be0e8647b4 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:00:03.617550 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8230b1-7410-4c52-8364-f5524e8c988d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:00:03.618165 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8230b1-7410-4c52-8364-f5524e8c988d None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:03.618495 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8230b1-7410-4c52-8364-f5524e8c988d None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:03.618766 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8230b1-7410-4c52-8364-f5524e8c988d None admin] PATH_INFO: `/v3/projects` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:03.619469 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8230b1-7410-4c52-8364-f5524e8c988d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:00:03.619771 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8230b1-7410-4c52-8364-f5524e8c988d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:00:03.621232 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8230b1-7410-4c52-8364-f5524e8c988d None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:00:03.638677 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 489/946] 10.222.0.55 () {56 vars in 1148 bytes} [Fri Sep 23 15:00:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:00:03.647067 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8230b1-7410-4c52-8364-f5524e8c988d None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:00:03.684159 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:00:03.717454 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054d45b4-aec3-47a9-bc4d-29be0e8647b4 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:00:03.763248 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88088b17-f01c-492c-bf1f-14bbd22b173f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:00:03.764709 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88088b17-f01c-492c-bf1f-14bbd22b173f None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:03.765189 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88088b17-f01c-492c-bf1f-14bbd22b173f None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:03.765658 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88088b17-f01c-492c-bf1f-14bbd22b173f None admin] PATH_INFO: `/v3/services` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:03.767052 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88088b17-f01c-492c-bf1f-14bbd22b173f None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:00:03.767581 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88088b17-f01c-492c-bf1f-14bbd22b173f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:00:03.769442 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:00:03.770069 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88088b17-f01c-492c-bf1f-14bbd22b173f None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:00:03.778861 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 458/947] 10.222.0.55 () {56 vars in 1122 bytes} [Fri Sep 23 15:00:03 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 15:00:03.779612 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8230b1-7410-4c52-8364-f5524e8c988d None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:00:03.784464 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88088b17-f01c-492c-bf1f-14bbd22b173f None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:00:03.817896 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23951b20-f05d-4100-97b6-19373b4f8315 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:00:03.818948 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23951b20-f05d-4100-97b6-19373b4f8315 None admin] REQUEST_METHOD: `PUT` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:03.819425 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23951b20-f05d-4100-97b6-19373b4f8315 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:03.819893 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23951b20-f05d-4100-97b6-19373b4f8315 None admin] PATH_INFO: `/v3/projects/87b3e9d4b6474bd4a04ba59ef52bf3e2/users/68206a6b4f244d45bf7bf1504396e430/roles/f2f7d48ece4f42f682922c1f1beefbd9` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:03.822988 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23951b20-f05d-4100-97b6-19373b4f8315 None admin] RBAC: Authorizing `identity:create_grant(project_id=87b3e9d4b6474bd4a04ba59ef52bf3e2, user_id=68206a6b4f244d45bf7bf1504396e430, role_id=f2f7d48ece4f42f682922c1f1beefbd9)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:00:03.824768 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23951b20-f05d-4100-97b6-19373b4f8315 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:00:03.897918 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88088b17-f01c-492c-bf1f-14bbd22b173f None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:00:03.933238 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 490/948] 10.222.0.55 () {58 vars in 1439 bytes} [Fri Sep 23 15:00:03 2022] PUT /identity/v3/projects/87b3e9d4b6474bd4a04ba59ef52bf3e2/users/68206a6b4f244d45bf7bf1504396e430/roles/f2f7d48ece4f42f682922c1f1beefbd9 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:00:03.937117 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab72d756-b00d-4f94-b4da-d8e157bdc477 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:00:03.937768 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab72d756-b00d-4f94-b4da-d8e157bdc477 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:03.938135 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab72d756-b00d-4f94-b4da-d8e157bdc477 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:03.938412 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab72d756-b00d-4f94-b4da-d8e157bdc477 None admin] PATH_INFO: `/v3/services` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:03.939122 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab72d756-b00d-4f94-b4da-d8e157bdc477 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:00:03.939430 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab72d756-b00d-4f94-b4da-d8e157bdc477 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:00:03.940940 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab72d756-b00d-4f94-b4da-d8e157bdc477 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:00:03.944011 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:00:03.948588 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 459/949] 10.222.0.55 () {56 vars in 1122 bytes} [Fri Sep 23 15:00:03 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:00:03.954257 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23951b20-f05d-4100-97b6-19373b4f8315 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:00:04.079394 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23951b20-f05d-4100-97b6-19373b4f8315 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:00:04.121506 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11820e1a-bf06-4488-802e-cca49e8a3bfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:00:04.122141 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11820e1a-bf06-4488-802e-cca49e8a3bfb None admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:04.122428 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11820e1a-bf06-4488-802e-cca49e8a3bfb None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:04.122708 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11820e1a-bf06-4488-802e-cca49e8a3bfb None admin] PATH_INFO: `/v3/endpoints` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:04.123406 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11820e1a-bf06-4488-802e-cca49e8a3bfb None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:00:04.124953 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11820e1a-bf06-4488-802e-cca49e8a3bfb None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:00:04.259521 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 491/950] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Sep 23 15:00:03 2022] POST /identity/v3/endpoints => generated 349 bytes in 306 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:00:04.542748 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbf2afd-57b0-413d-8ffe-36af7ee30a6d None None] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:04.543257 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbf2afd-57b0-413d-8ffe-36af7ee30a6d None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:04.543333 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbf2afd-57b0-413d-8ffe-36af7ee30a6d None None] PATH_INFO: `/` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:04.546619 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 460/951] 10.222.0.55 () {54 vars in 927 bytes} [Fri Sep 23 15:00:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:00:04.552218 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e531e133-56d7-4918-a6fe-c0845cbab938 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:04.552504 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e531e133-56d7-4918-a6fe-c0845cbab938 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:04.552776 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e531e133-56d7-4918-a6fe-c0845cbab938 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:04.576177 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:00:04.584129 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-e531e133-56d7-4918-a6fe-c0845cbab938 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:00:04.592082 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-e531e133-56d7-4918-a6fe-c0845cbab938 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:00:04.679469 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 492/952] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Sep 23 15:00:04 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 23 15:00:04.967259 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbf2afd-57b0-413d-8ffe-36af7ee30a6d None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:00:05.110696 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbf2afd-57b0-413d-8ffe-36af7ee30a6d None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:00:05.146234 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3916e4-85ce-4bc8-b875-9838657d6251 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:00:05.146870 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3916e4-85ce-4bc8-b875-9838657d6251 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:05.147145 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3916e4-85ce-4bc8-b875-9838657d6251 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:05.147418 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3916e4-85ce-4bc8-b875-9838657d6251 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:05.148007 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3916e4-85ce-4bc8-b875-9838657d6251 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:00:05.149444 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3916e4-85ce-4bc8-b875-9838657d6251 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:00:05.158968 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application [None req-cc3916e4-85ce-4bc8-b875-9838657d6251 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 23 15:00:05.158968 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:00:05.158968 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 23 15:00:05.158968 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 23 15:00:05.158968 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 23 15:00:05.158968 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 23 15:00:05.158968 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 23 15:00:05.158968 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 23 15:00:05.158968 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 23 15:00:05.158968 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Sep 23 15:00:05.158968 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 23 15:00:05.158968 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Sep 23 15:00:05.158968 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:00:05.158968 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:00:05.158968 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:00:05.158968 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:00:05.158968 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:00:05.160035 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:00:05.160035 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:00:05.160035 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:00:05.160035 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:00:05.160035 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:00:05.160035 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:00:05.160035 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 23 15:00:05.160035 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 23 15:00:05.160035 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 23 15:00:05.160035 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 23 15:00:05.160035 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:00:05.160035 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:00:05.160035 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:00:05.160035 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:00:05.160035 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:00:05.160035 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:00:05.160898 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:00:05.160898 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application with Lock( Sep 23 15:00:05.160898 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:00:05.160898 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:00:05.160898 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:00:05.160898 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:00:05.160898 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:00:05.160898 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:00:05.160898 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:00:05.160898 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:00:05.160898 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 23 15:00:05.160898 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 23 15:00:05.160898 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 23 15:00:05.160898 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Sep 23 15:00:05.160898 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 461/953] 10.222.0.55 () {56 vars in 1111 bytes} [Fri Sep 23 15:00:04 2022] GET /identity/v3/services/compute => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:00:05.166602 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e531e133-56d7-4918-a6fe-c0845cbab938 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:00:05.266996 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e531e133-56d7-4918-a6fe-c0845cbab938 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:00:05.297274 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:00:05.302301 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c473012-f5a0-4f23-acca-c6436a2debc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:00:05.302911 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c473012-f5a0-4f23-acca-c6436a2debc1 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:05.303184 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c473012-f5a0-4f23-acca-c6436a2debc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:05.303453 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c473012-f5a0-4f23-acca-c6436a2debc1 None admin] PATH_INFO: `/v3/services` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:05.304108 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c473012-f5a0-4f23-acca-c6436a2debc1 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:00:05.304405 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c473012-f5a0-4f23-acca-c6436a2debc1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:00:05.306275 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c473012-f5a0-4f23-acca-c6436a2debc1 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:00:05.311517 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 493/954] 10.222.0.55 () {56 vars in 1112 bytes} [Fri Sep 23 15:00:05 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:00:05.317058 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3916e4-85ce-4bc8-b875-9838657d6251 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:00:05.387306 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:00:05.426738 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3916e4-85ce-4bc8-b875-9838657d6251 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:00:05.467690 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f31f96-f124-466c-9f1f-17cef1d8d4b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:00:05.468406 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f31f96-f124-466c-9f1f-17cef1d8d4b2 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:05.468623 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f31f96-f124-466c-9f1f-17cef1d8d4b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:05.468894 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f31f96-f124-466c-9f1f-17cef1d8d4b2 None admin] PATH_INFO: `/v3/services` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:05.469518 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f31f96-f124-466c-9f1f-17cef1d8d4b2 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:00:05.469839 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f31f96-f124-466c-9f1f-17cef1d8d4b2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:00:05.471208 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f31f96-f124-466c-9f1f-17cef1d8d4b2 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:00:05.476150 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 462/955] 10.222.0.55 () {56 vars in 1112 bytes} [Fri Sep 23 15:00:05 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 15:00:05.832142 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2b3d59-329a-48c2-9aa0-e069ec1ee941 None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:05.832643 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2b3d59-329a-48c2-9aa0-e069ec1ee941 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:05.833048 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2b3d59-329a-48c2-9aa0-e069ec1ee941 None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:05.834721 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 494/956] 10.222.0.55 () {54 vars in 927 bytes} [Fri Sep 23 15:00:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:00:05.840835 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c879f71-c850-4852-b8fc-8f945ea74764 None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:05.841296 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c879f71-c850-4852-b8fc-8f945ea74764 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:05.841563 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c879f71-c850-4852-b8fc-8f945ea74764 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:05.860707 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-7c879f71-c850-4852-b8fc-8f945ea74764 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:00:05.861946 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-7c879f71-c850-4852-b8fc-8f945ea74764 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:00:05.933263 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 463/957] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Sep 23 15:00:05 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 23 15:00:06.205073 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2b3d59-329a-48c2-9aa0-e069ec1ee941 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:00:06.315291 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2b3d59-329a-48c2-9aa0-e069ec1ee941 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:00:06.352128 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fd403b-6e89-4ef9-af55-4404d86f409b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:00:06.352780 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fd403b-6e89-4ef9-af55-4404d86f409b None admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:06.353063 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fd403b-6e89-4ef9-af55-4404d86f409b None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:06.353336 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fd403b-6e89-4ef9-af55-4404d86f409b None admin] PATH_INFO: `/v3/projects` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:06.353991 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fd403b-6e89-4ef9-af55-4404d86f409b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:00:06.355415 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fd403b-6e89-4ef9-af55-4404d86f409b None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:00:06.358049 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: WARNING keystone.server.flask.common [None req-73fd403b-6e89-4ef9-af55-4404d86f409b None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 23 15:00:06.374556 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164ec86e-7389-4b47-8ac4-bf037b83cbab None None] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:06.375156 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164ec86e-7389-4b47-8ac4-bf037b83cbab None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:06.375434 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164ec86e-7389-4b47-8ac4-bf037b83cbab None None] PATH_INFO: `/` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:06.376839 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 464/958] 10.222.0.55 () {54 vars in 927 bytes} [Fri Sep 23 15:00:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:00:06.382799 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e8e08e-ce0e-45eb-a39d-0c5c49fbf5d1 None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:06.383103 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e8e08e-ce0e-45eb-a39d-0c5c49fbf5d1 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:06.383399 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e8e08e-ce0e-45eb-a39d-0c5c49fbf5d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:06.403146 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-48e8e08e-ce0e-45eb-a39d-0c5c49fbf5d1 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:00:06.404382 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-48e8e08e-ce0e-45eb-a39d-0c5c49fbf5d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:00:06.423383 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.sql.core [None req-73fd403b-6e89-4ef9-af55-4404d86f409b None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 23 15:00:06.423383 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 23 15:00:06.423383 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [parameters: {'id': 'dfb4d5657d04445eb238362adeaccfd8', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 23 15:00:06.423383 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81994) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 23 15:00:06.430816 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-73fd403b-6e89-4ef9-af55-4404d86f409b None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id dfb4d5657d04445eb238362adeaccfd8.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id dfb4d5657d04445eb238362adeaccfd8. Sep 23 15:00:06.430816 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:00:06.430816 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 23 15:00:06.430816 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 23 15:00:06.430816 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 23 15:00:06.430816 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 23 15:00:06.430816 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 23 15:00:06.430816 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application result = self._query(query) Sep 23 15:00:06.430816 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 23 15:00:06.430816 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application conn.query(q) Sep 23 15:00:06.430816 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 23 15:00:06.430816 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 23 15:00:06.430816 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 23 15:00:06.431719 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application result.read() Sep 23 15:00:06.431719 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 23 15:00:06.431719 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 23 15:00:06.431719 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 23 15:00:06.431719 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 23 15:00:06.431719 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 23 15:00:06.431719 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 23 15:00:06.431719 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 23 15:00:06.431719 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 23 15:00:06.431719 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 23 15:00:06.431719 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Sep 23 15:00:06.431719 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 23 15:00:06.431719 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Sep 23 15:00:06.431719 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:00:06.431719 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 23 15:00:06.431719 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 23 15:00:06.431719 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Sep 23 15:00:06.431719 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return project_ref.to_dict() Sep 23 15:00:06.433425 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 23 15:00:06.433425 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application next(self.gen) Sep 23 15:00:06.433425 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 23 15:00:06.433425 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application yield resource Sep 23 15:00:06.433425 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 23 15:00:06.433425 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application next(self.gen) Sep 23 15:00:06.433425 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 23 15:00:06.433425 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application self.session.rollback() Sep 23 15:00:06.433425 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 23 15:00:06.433425 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application self.force_reraise() Sep 23 15:00:06.433425 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 23 15:00:06.433425 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise self.value Sep 23 15:00:06.433425 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 23 15:00:06.433425 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 23 15:00:06.433425 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 23 15:00:06.433425 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application session.commit() Sep 23 15:00:06.433425 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 23 15:00:06.434344 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 23 15:00:06.434344 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 23 15:00:06.434344 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application self._prepare_impl() Sep 23 15:00:06.434344 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 23 15:00:06.434344 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application self.session.flush() Sep 23 15:00:06.434344 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 23 15:00:06.434344 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application self._flush(objects) Sep 23 15:00:06.434344 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 23 15:00:06.434344 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 23 15:00:06.434344 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 23 15:00:06.434344 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application compat.raise_( Sep 23 15:00:06.434344 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 23 15:00:06.434344 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception Sep 23 15:00:06.434344 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 23 15:00:06.434344 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application flush_context.execute() Sep 23 15:00:06.434344 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 23 15:00:06.434344 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rec.execute(self) Sep 23 15:00:06.435290 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 23 15:00:06.435290 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 23 15:00:06.435290 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 23 15:00:06.435290 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 23 15:00:06.435290 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 23 15:00:06.435290 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 23 15:00:06.435290 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 23 15:00:06.435290 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 23 15:00:06.435290 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 23 15:00:06.435290 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 23 15:00:06.435290 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 23 15:00:06.435290 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 23 15:00:06.435290 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 23 15:00:06.435290 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 23 15:00:06.435290 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 23 15:00:06.436833 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 23 15:00:06.436833 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 23 15:00:06.436833 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception Sep 23 15:00:06.436833 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 23 15:00:06.436833 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 23 15:00:06.436833 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 23 15:00:06.436833 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 23 15:00:06.436833 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 23 15:00:06.436833 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application result = self._query(query) Sep 23 15:00:06.436833 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 23 15:00:06.436833 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application conn.query(q) Sep 23 15:00:06.436833 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 23 15:00:06.436833 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 23 15:00:06.436833 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 23 15:00:06.436833 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application result.read() Sep 23 15:00:06.436833 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 23 15:00:06.436833 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 23 15:00:06.438160 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 23 15:00:06.438160 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 23 15:00:06.438160 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 23 15:00:06.438160 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 23 15:00:06.438160 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 23 15:00:06.438160 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 23 15:00:06.438160 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 23 15:00:06.438160 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 23 15:00:06.438160 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [parameters: {'id': 'dfb4d5657d04445eb238362adeaccfd8', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 23 15:00:06.438160 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 23 15:00:06.438160 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Sep 23 15:00:06.438160 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 23 15:00:06.438160 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Sep 23 15:00:06.438160 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:00:06.438160 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Sep 23 15:00:06.439265 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Sep 23 15:00:06.439265 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 23 15:00:06.439265 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 23 15:00:06.439265 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Sep 23 15:00:06.439265 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Sep 23 15:00:06.439265 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 23 15:00:06.439265 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Sep 23 15:00:06.439265 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:00:06.439265 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:00:06.439265 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:00:06.439265 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:00:06.439265 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:00:06.439265 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:00:06.439265 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:00:06.439265 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:00:06.439265 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:00:06.439265 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:00:06.440816 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:00:06.440816 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Sep 23 15:00:06.440816 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Sep 23 15:00:06.440816 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:00:06.440816 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:00:06.440816 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Sep 23 15:00:06.440816 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.Conflict( Sep 23 15:00:06.440816 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id dfb4d5657d04445eb238362adeaccfd8. Sep 23 15:00:06.440816 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Sep 23 15:00:06.440816 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 495/959] 10.222.0.55 () {60 vars in 1140 bytes} [Fri Sep 23 15:00:06 2022] POST /identity/v3/projects => generated 262 bytes in 228 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:00:06.442525 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fd403b-6e89-4ef9-af55-4404d86f409b None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:00:06.483726 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:00:06.500638 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 465/960] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Sep 23 15:00:06 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 23 15:00:06.513250 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:00:06.563197 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fd403b-6e89-4ef9-af55-4404d86f409b None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:00:06.599178 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762a1c29-1b17-42c8-8325-99f0b466bfdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:00:06.599850 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762a1c29-1b17-42c8-8325-99f0b466bfdb None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:06.600211 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762a1c29-1b17-42c8-8325-99f0b466bfdb None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:06.601017 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762a1c29-1b17-42c8-8325-99f0b466bfdb None admin] PATH_INFO: `/v3/projects/service` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:06.608855 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762a1c29-1b17-42c8-8325-99f0b466bfdb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:00:06.610761 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762a1c29-1b17-42c8-8325-99f0b466bfdb None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:00:06.624898 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-762a1c29-1b17-42c8-8325-99f0b466bfdb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 15:00:06.624898 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:00:06.624898 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:00:06.624898 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:00:06.624898 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:00:06.624898 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:00:06.624898 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:00:06.624898 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:00:06.624898 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:00:06.624898 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:00:06.624898 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:00:06.624898 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:00:06.624898 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 23 15:00:06.624898 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 23 15:00:06.624898 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 23 15:00:06.625993 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 23 15:00:06.625993 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:00:06.625993 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:00:06.625993 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:00:06.625993 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:00:06.625993 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:00:06.625993 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:00:06.625993 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:00:06.625993 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Sep 23 15:00:06.625993 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:00:06.625993 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:00:06.625993 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:00:06.625993 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:00:06.625993 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:00:06.625993 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:00:06.625993 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:00:06.625993 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:00:06.626979 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 23 15:00:06.626979 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 23 15:00:06.626979 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 23 15:00:06.626979 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 23 15:00:06.626979 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 23 15:00:06.626979 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 23 15:00:06.626979 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 15:00:06.626979 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Sep 23 15:00:06.633795 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 496/961] 10.222.0.55 () {56 vars in 1111 bytes} [Fri Sep 23 15:00:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 23 15:00:06.640322 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e8e08e-ce0e-45eb-a39d-0c5c49fbf5d1 None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:00:06.738507 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d889c9-d9eb-4217-b1eb-cbdf19363c59 None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:06.738807 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d889c9-d9eb-4217-b1eb-cbdf19363c59 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:06.739011 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d889c9-d9eb-4217-b1eb-cbdf19363c59 None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:06.747996 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 497/962] 10.222.0.55 () {54 vars in 927 bytes} [Fri Sep 23 15:00:06 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:00:06.749748 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42268b5-a480-4e60-ab80-f48c43ad434d None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:06.750110 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42268b5-a480-4e60-ab80-f48c43ad434d None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:06.750435 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e8e08e-ce0e-45eb-a39d-0c5c49fbf5d1 None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:00:06.750547 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42268b5-a480-4e60-ab80-f48c43ad434d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:06.772121 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-b42268b5-a480-4e60-ab80-f48c43ad434d None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:00:06.774199 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-b42268b5-a480-4e60-ab80-f48c43ad434d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:00:06.789213 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd2bb1e-6182-45e8-a48d-7428aed0e0e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:00:06.789830 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd2bb1e-6182-45e8-a48d-7428aed0e0e3 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:06.790106 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd2bb1e-6182-45e8-a48d-7428aed0e0e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:06.790530 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd2bb1e-6182-45e8-a48d-7428aed0e0e3 None admin] PATH_INFO: `/v3/projects` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:06.791200 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd2bb1e-6182-45e8-a48d-7428aed0e0e3 None admin] RBAC: Adding query filter params (name=service) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:00:06.791517 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd2bb1e-6182-45e8-a48d-7428aed0e0e3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:00:06.792976 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd2bb1e-6182-45e8-a48d-7428aed0e0e3 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:00:06.811429 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 466/963] 10.222.0.55 () {56 vars in 1112 bytes} [Fri Sep 23 15:00:06 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:00:06.818509 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd2bb1e-6182-45e8-a48d-7428aed0e0e3 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:00:06.870395 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 498/964] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Sep 23 15:00:06 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 23 15:00:06.882370 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:00:06.940925 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd2bb1e-6182-45e8-a48d-7428aed0e0e3 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:00:06.981175 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600ddca9-42fd-4c3f-99c2-c4006e2be8dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:00:06.981843 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600ddca9-42fd-4c3f-99c2-c4006e2be8dd None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:06.982121 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600ddca9-42fd-4c3f-99c2-c4006e2be8dd None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:06.982389 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600ddca9-42fd-4c3f-99c2-c4006e2be8dd None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:06.994863 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600ddca9-42fd-4c3f-99c2-c4006e2be8dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 15:00:06.999710 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application [None req-600ddca9-42fd-4c3f-99c2-c4006e2be8dd None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 23 15:00:06.999710 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:00:06.999710 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:00:06.999710 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:00:06.999710 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:00:06.999710 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:00:06.999710 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:00:06.999710 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:00:06.999710 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:00:06.999710 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:00:06.999710 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:00:06.999710 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:00:06.999710 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 15:00:06.999710 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 15:00:06.999710 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 15:00:06.999710 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 15:00:07.000731 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 15:00:07.000731 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 15:00:07.000731 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 15:00:07.000731 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 15:00:07.000731 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:00:07.000731 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:00:07.000731 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:00:07.000731 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:00:07.000731 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:00:07.000731 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:00:07.000731 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:00:07.000731 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application with Lock( Sep 23 15:00:07.000731 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:00:07.000731 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:00:07.000731 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:00:07.000731 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:00:07.001596 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:00:07.001596 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:00:07.001596 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:00:07.001596 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:00:07.001596 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 15:00:07.001596 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 15:00:07.001596 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 15:00:07.001596 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 15:00:07.001596 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 15:00:07.001596 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 15:00:07.001596 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 23 15:00:07.001596 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Sep 23 15:00:07.001596 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 467/965] 10.222.0.55 () {56 vars in 1120 bytes} [Fri Sep 23 15:00:06 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 23 15:00:07.008113 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42268b5-a480-4e60-ab80-f48c43ad434d None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:00:07.056264 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:00:07.123416 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42268b5-a480-4e60-ab80-f48c43ad434d None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:00:07.161774 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600ddca9-42fd-4c3f-99c2-c4006e2be8dd None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:00:07.163185 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a527645-ddf9-4401-a4e5-1fd974614585 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:00:07.163810 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a527645-ddf9-4401-a4e5-1fd974614585 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:07.164125 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a527645-ddf9-4401-a4e5-1fd974614585 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:07.164396 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a527645-ddf9-4401-a4e5-1fd974614585 None admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:07.165071 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a527645-ddf9-4401-a4e5-1fd974614585 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:00:07.165905 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a527645-ddf9-4401-a4e5-1fd974614585 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:00:07.167279 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a527645-ddf9-4401-a4e5-1fd974614585 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:00:07.176593 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 499/966] 10.222.0.55 () {56 vars in 1115 bytes} [Fri Sep 23 15:00:07 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:00:07.183089 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a527645-ddf9-4401-a4e5-1fd974614585 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:00:07.295890 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a527645-ddf9-4401-a4e5-1fd974614585 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:00:07.300109 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:00:07.301485 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600ddca9-42fd-4c3f-99c2-c4006e2be8dd None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:00:07.313994 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:00:07.340342 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b31dd00-bd81-4b5c-a665-50ec0e9f7459 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:00:07.341293 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b31dd00-bd81-4b5c-a665-50ec0e9f7459 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:07.341655 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b31dd00-bd81-4b5c-a665-50ec0e9f7459 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:07.341996 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b31dd00-bd81-4b5c-a665-50ec0e9f7459 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:07.346334 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adc072f-672d-4be9-b4f9-cd367f39ed4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:00:07.347256 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adc072f-672d-4be9-b4f9-cd367f39ed4f None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:07.356573 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adc072f-672d-4be9-b4f9-cd367f39ed4f None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:07.356573 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adc072f-672d-4be9-b4f9-cd367f39ed4f None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:07.356573 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2adc072f-672d-4be9-b4f9-cd367f39ed4f None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:00:07.356573 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2adc072f-672d-4be9-b4f9-cd367f39ed4f None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:00:07.356886 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b31dd00-bd81-4b5c-a665-50ec0e9f7459 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:00:07.356973 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b31dd00-bd81-4b5c-a665-50ec0e9f7459 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:00:07.361960 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application [None req-2adc072f-672d-4be9-b4f9-cd367f39ed4f None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 23 15:00:07.361960 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:00:07.361960 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 23 15:00:07.361960 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 23 15:00:07.361960 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 23 15:00:07.361960 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 23 15:00:07.361960 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 23 15:00:07.361960 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 23 15:00:07.361960 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 23 15:00:07.361960 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Sep 23 15:00:07.361960 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 23 15:00:07.361960 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Sep 23 15:00:07.361960 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:00:07.361960 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:00:07.361960 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:00:07.361960 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:00:07.361960 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:00:07.363040 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:00:07.363040 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:00:07.363040 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:00:07.363040 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:00:07.363040 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:00:07.363040 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:00:07.363040 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 23 15:00:07.363040 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 23 15:00:07.363040 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 23 15:00:07.363040 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 23 15:00:07.363040 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:00:07.363040 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:00:07.363040 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:00:07.363040 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:00:07.363040 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:00:07.363040 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:00:07.363899 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:00:07.363899 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application with Lock( Sep 23 15:00:07.363899 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:00:07.363899 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:00:07.363899 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:00:07.363899 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:00:07.363899 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:00:07.363899 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:00:07.363899 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:00:07.363899 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:00:07.363899 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 23 15:00:07.363899 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 23 15:00:07.363899 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 23 15:00:07.363899 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Sep 23 15:00:07.365604 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 468/967] 10.222.0.55 () {56 vars in 1126 bytes} [Fri Sep 23 15:00:07 2022] GET /identity/v3/services/object-store => generated 93 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:00:07.371279 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adc072f-672d-4be9-b4f9-cd367f39ed4f None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:00:07.376727 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-6b31dd00-bd81-4b5c-a665-50ec0e9f7459 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 15:00:07.376727 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:00:07.376727 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:00:07.376727 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:00:07.376727 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:00:07.376727 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:00:07.376727 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:00:07.376727 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:00:07.376727 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:00:07.376727 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:00:07.376727 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:00:07.376727 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:00:07.376727 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 15:00:07.376727 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 15:00:07.376727 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 15:00:07.376727 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 15:00:07.377666 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:00:07.377666 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:00:07.377666 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:00:07.377666 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:00:07.377666 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:00:07.377666 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:00:07.377666 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:00:07.377666 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Sep 23 15:00:07.377666 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:00:07.377666 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:00:07.377666 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:00:07.377666 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:00:07.377666 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:00:07.377666 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:00:07.377666 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:00:07.377666 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:00:07.377666 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 15:00:07.378668 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 15:00:07.378668 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 15:00:07.378668 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Sep 23 15:00:07.380595 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 500/968] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Sep 23 15:00:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 23 15:00:07.386655 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b31dd00-bd81-4b5c-a665-50ec0e9f7459 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:00:07.501974 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adc072f-672d-4be9-b4f9-cd367f39ed4f None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:00:07.503371 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b31dd00-bd81-4b5c-a665-50ec0e9f7459 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:00:07.545374 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec60a016-6aa3-4dde-ad65-38fb3d80ae41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:00:07.545626 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cc229e-44a5-4992-8462-369a9963d4b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:00:07.546198 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec60a016-6aa3-4dde-ad65-38fb3d80ae41 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:07.546471 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec60a016-6aa3-4dde-ad65-38fb3d80ae41 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:07.546809 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec60a016-6aa3-4dde-ad65-38fb3d80ae41 None admin] PATH_INFO: `/v3/domains` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:07.547109 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cc229e-44a5-4992-8462-369a9963d4b7 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:07.547573 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cc229e-44a5-4992-8462-369a9963d4b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:07.547677 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec60a016-6aa3-4dde-ad65-38fb3d80ae41 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:00:07.547961 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec60a016-6aa3-4dde-ad65-38fb3d80ae41 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:00:07.548317 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cc229e-44a5-4992-8462-369a9963d4b7 None admin] PATH_INFO: `/v3/services` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:07.549115 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cc229e-44a5-4992-8462-369a9963d4b7 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:00:07.549575 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec60a016-6aa3-4dde-ad65-38fb3d80ae41 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:00:07.549786 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cc229e-44a5-4992-8462-369a9963d4b7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:00:07.551509 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cc229e-44a5-4992-8462-369a9963d4b7 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:00:07.557610 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 469/969] 10.222.0.55 () {56 vars in 1122 bytes} [Fri Sep 23 15:00:07 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 15:00:07.563833 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cc229e-44a5-4992-8462-369a9963d4b7 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:00:07.574014 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 501/970] 10.222.0.55 () {56 vars in 1109 bytes} [Fri Sep 23 15:00:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:00:07.575930 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec60a016-6aa3-4dde-ad65-38fb3d80ae41 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:00:07.596804 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:00:07.609714 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:00:07.701925 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cc229e-44a5-4992-8462-369a9963d4b7 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:00:07.707627 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec60a016-6aa3-4dde-ad65-38fb3d80ae41 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:00:07.746453 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27bcf09-1f9d-4179-a520-a45aa7ff31d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:00:07.747189 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27bcf09-1f9d-4179-a520-a45aa7ff31d6 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:07.747535 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27bcf09-1f9d-4179-a520-a45aa7ff31d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:07.747848 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27bcf09-1f9d-4179-a520-a45aa7ff31d6 None admin] PATH_INFO: `/v3/services` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:07.748872 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27bcf09-1f9d-4179-a520-a45aa7ff31d6 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:00:07.749238 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27bcf09-1f9d-4179-a520-a45aa7ff31d6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:00:07.749864 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d97818f-037c-470f-b7d9-024289dc0592 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:00:07.750501 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d97818f-037c-470f-b7d9-024289dc0592 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:07.750597 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27bcf09-1f9d-4179-a520-a45aa7ff31d6 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:00:07.750834 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d97818f-037c-470f-b7d9-024289dc0592 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:07.751109 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d97818f-037c-470f-b7d9-024289dc0592 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:07.756220 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 470/971] 10.222.0.55 () {56 vars in 1122 bytes} [Fri Sep 23 15:00:07 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 15:00:07.758570 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d97818f-037c-470f-b7d9-024289dc0592 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:00:07.760780 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d97818f-037c-470f-b7d9-024289dc0592 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:00:07.762650 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1589447c-3cfd-4922-a9cc-f569fd0d4de5 None None] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:07.764709 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1589447c-3cfd-4922-a9cc-f569fd0d4de5 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:07.764709 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1589447c-3cfd-4922-a9cc-f569fd0d4de5 None None] PATH_INFO: `/` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:07.768088 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 471/972] 10.222.0.55 () {54 vars in 927 bytes} [Fri Sep 23 15:00:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:00:07.768088 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1589447c-3cfd-4922-a9cc-f569fd0d4de5 None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:00:07.773293 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-6d97818f-037c-470f-b7d9-024289dc0592 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 23 15:00:07.773293 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:00:07.773293 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:00:07.773293 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:00:07.773293 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:00:07.773293 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:00:07.773293 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:00:07.773293 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:00:07.773293 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:00:07.773293 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:00:07.773293 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:00:07.773293 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:00:07.773293 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 23 15:00:07.773293 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 23 15:00:07.773293 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 23 15:00:07.773293 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 23 15:00:07.774372 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:00:07.774372 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:00:07.774372 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 23 15:00:07.774372 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 15:00:07.774372 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 23 15:00:07.774372 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 15:00:07.774372 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:00:07.774372 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:00:07.774372 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:00:07.774372 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:00:07.774372 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:00:07.774372 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Sep 23 15:00:07.774372 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:00:07.774372 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:00:07.774372 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:00:07.774372 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:00:07.774372 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:00:07.774372 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:00:07.776395 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:00:07.776395 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:00:07.776395 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 23 15:00:07.776395 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 23 15:00:07.776395 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 23 15:00:07.776395 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 23 15:00:07.776395 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 23 15:00:07.776395 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 23 15:00:07.776395 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 23 15:00:07.776395 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Sep 23 15:00:07.776395 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 502/973] 10.222.0.55 () {56 vars in 1117 bytes} [Fri Sep 23 15:00:07 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:00:07.777017 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410d0535-2cc7-4861-993d-b3909c158001 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:07.777017 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410d0535-2cc7-4861-993d-b3909c158001 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:07.777142 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410d0535-2cc7-4861-993d-b3909c158001 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:07.796418 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-410d0535-2cc7-4861-993d-b3909c158001 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:00:07.797530 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-410d0535-2cc7-4861-993d-b3909c158001 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:00:07.872915 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 503/974] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Sep 23 15:00:07 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 23 15:00:07.874720 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410d0535-2cc7-4861-993d-b3909c158001 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:00:07.883254 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:00:07.893284 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:00:07.897824 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1589447c-3cfd-4922-a9cc-f569fd0d4de5 None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:00:07.934926 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4be91b-c339-4a84-84ba-dbd4dd4d7843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:00:07.936216 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4be91b-c339-4a84-84ba-dbd4dd4d7843 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:07.936748 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4be91b-c339-4a84-84ba-dbd4dd4d7843 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:07.937202 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4be91b-c339-4a84-84ba-dbd4dd4d7843 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:07.938092 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4be91b-c339-4a84-84ba-dbd4dd4d7843 None admin] RBAC: Adding query filter params (interface=admin, service_id=c7b748108d7d4bfe9fad4a8d8a9e6fc3, region_id=RegionOne) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:00:07.938612 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4be91b-c339-4a84-84ba-dbd4dd4d7843 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:00:07.940377 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4be91b-c339-4a84-84ba-dbd4dd4d7843 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:00:07.946192 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 472/975] 10.222.0.55 () {56 vars in 1249 bytes} [Fri Sep 23 15:00:07 2022] GET /identity/v3/endpoints?service_id=c7b748108d7d4bfe9fad4a8d8a9e6fc3&interface=admin®ion_id=RegionOne => generated 193 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:00:07.957153 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4be91b-c339-4a84-84ba-dbd4dd4d7843 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:00:07.992755 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410d0535-2cc7-4861-993d-b3909c158001 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:00:08.034270 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ec6e71-dd1c-477e-af1a-751e198e7c92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:00:08.035413 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ec6e71-dd1c-477e-af1a-751e198e7c92 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:08.035871 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ec6e71-dd1c-477e-af1a-751e198e7c92 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:08.036247 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ec6e71-dd1c-477e-af1a-751e198e7c92 None admin] PATH_INFO: `/v3/users` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:08.037011 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ec6e71-dd1c-477e-af1a-751e198e7c92 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:00:08.037320 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ec6e71-dd1c-477e-af1a-751e198e7c92 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:00:08.039278 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ec6e71-dd1c-477e-af1a-751e198e7c92 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:00:08.068235 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 504/976] 10.222.0.55 () {56 vars in 1149 bytes} [Fri Sep 23 15:00:07 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:00:08.069442 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ec6e71-dd1c-477e-af1a-751e198e7c92 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:00:08.103590 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4be91b-c339-4a84-84ba-dbd4dd4d7843 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:00:08.144451 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a142480-4278-4575-807d-ba65780ec9cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:00:08.145193 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a142480-4278-4575-807d-ba65780ec9cb None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:08.145504 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a142480-4278-4575-807d-ba65780ec9cb None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:08.145783 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a142480-4278-4575-807d-ba65780ec9cb None admin] PATH_INFO: `/v3/services` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:08.146438 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a142480-4278-4575-807d-ba65780ec9cb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:00:08.146970 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:00:08.148746 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a142480-4278-4575-807d-ba65780ec9cb None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:00:08.156029 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 473/977] 10.222.0.55 () {56 vars in 1087 bytes} [Fri Sep 23 15:00:07 2022] GET /identity/v3/services => generated 1497 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 23 15:00:08.158685 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a142480-4278-4575-807d-ba65780ec9cb None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:00:08.198152 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ec6e71-dd1c-477e-af1a-751e198e7c92 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:00:08.208102 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:00:08.239212 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539fe64d-243d-47b7-aab5-32952d65395d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:00:08.240008 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539fe64d-243d-47b7-aab5-32952d65395d None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:08.240430 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539fe64d-243d-47b7-aab5-32952d65395d None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:08.240797 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539fe64d-243d-47b7-aab5-32952d65395d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:08.321592 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a142480-4278-4575-807d-ba65780ec9cb None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:00:08.323619 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539fe64d-243d-47b7-aab5-32952d65395d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:00:08.325258 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539fe64d-243d-47b7-aab5-32952d65395d None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:00:08.373117 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f78b938-e5e9-4256-ae77-c1991d3c0886 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:00:08.373783 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f78b938-e5e9-4256-ae77-c1991d3c0886 None admin] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:08.374059 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f78b938-e5e9-4256-ae77-c1991d3c0886 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:08.374781 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f78b938-e5e9-4256-ae77-c1991d3c0886 None admin] PATH_INFO: `/v3/services` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:08.375433 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f78b938-e5e9-4256-ae77-c1991d3c0886 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:00:08.377362 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f78b938-e5e9-4256-ae77-c1991d3c0886 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:00:08.401488 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:00:08.440296 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 505/978] 10.222.0.55 () {58 vars in 1282 bytes} [Fri Sep 23 15:00:08 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 23 15:00:08.446050 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539fe64d-243d-47b7-aab5-32952d65395d None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:00:08.503300 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 474/979] 10.222.0.55 () {60 vars in 1141 bytes} [Fri Sep 23 15:00:08 2022] POST /identity/v3/services => generated 242 bytes in 345 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:00:08.560755 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539fe64d-243d-47b7-aab5-32952d65395d None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:00:08.612916 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6d29ec-96fe-41dd-b0aa-d3e0fa3ca73a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:00:08.612916 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6d29ec-96fe-41dd-b0aa-d3e0fa3ca73a None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:08.612916 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6d29ec-96fe-41dd-b0aa-d3e0fa3ca73a None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:08.612916 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6d29ec-96fe-41dd-b0aa-d3e0fa3ca73a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:08.623207 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6d29ec-96fe-41dd-b0aa-d3e0fa3ca73a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:00:08.624770 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6d29ec-96fe-41dd-b0aa-d3e0fa3ca73a None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:00:08.642035 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-5b6d29ec-96fe-41dd-b0aa-d3e0fa3ca73a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 15:00:08.642035 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:00:08.642035 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:00:08.642035 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:00:08.642035 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:00:08.642035 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:00:08.642035 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:00:08.642035 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:00:08.642035 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:00:08.642035 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:00:08.642035 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:00:08.642035 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:00:08.642035 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 15:00:08.642035 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 15:00:08.642035 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 15:00:08.642035 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 15:00:08.643064 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:00:08.643064 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:00:08.643064 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:00:08.643064 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:00:08.643064 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:00:08.643064 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:00:08.643064 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:00:08.643064 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Sep 23 15:00:08.643064 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:00:08.643064 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:00:08.643064 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:00:08.643064 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:00:08.643064 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:00:08.643064 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:00:08.643064 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:00:08.643064 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:00:08.643064 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 15:00:08.644013 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 15:00:08.644013 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 15:00:08.644013 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Sep 23 15:00:08.644013 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 506/980] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Sep 23 15:00:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 23 15:00:08.650915 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f78b938-e5e9-4256-ae77-c1991d3c0886 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:00:08.758685 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:00:08.771965 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f78b938-e5e9-4256-ae77-c1991d3c0886 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:00:08.810405 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14709db5-b908-4da6-9536-27a92cbfd195 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:00:08.811026 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14709db5-b908-4da6-9536-27a92cbfd195 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:08.811595 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14709db5-b908-4da6-9536-27a92cbfd195 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:08.811865 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14709db5-b908-4da6-9536-27a92cbfd195 None admin] PATH_INFO: `/v3/domains` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:08.812637 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14709db5-b908-4da6-9536-27a92cbfd195 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:00:08.812936 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14709db5-b908-4da6-9536-27a92cbfd195 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:00:08.814555 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14709db5-b908-4da6-9536-27a92cbfd195 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:00:08.827407 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 475/981] 10.222.0.55 () {56 vars in 1109 bytes} [Fri Sep 23 15:00:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:00:08.833025 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6d29ec-96fe-41dd-b0aa-d3e0fa3ca73a None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:00:08.849757 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:00:08.941369 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6d29ec-96fe-41dd-b0aa-d3e0fa3ca73a None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:00:08.977291 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b5fa01-f81d-422a-8ad8-e1a91957a74e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:00:08.978457 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b5fa01-f81d-422a-8ad8-e1a91957a74e None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:08.978751 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b5fa01-f81d-422a-8ad8-e1a91957a74e None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:08.978990 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b5fa01-f81d-422a-8ad8-e1a91957a74e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:08.985976 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b5fa01-f81d-422a-8ad8-e1a91957a74e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:00:08.987402 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b5fa01-f81d-422a-8ad8-e1a91957a74e None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:00:08.998925 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-c0b5fa01-f81d-422a-8ad8-e1a91957a74e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 15:00:08.998925 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:00:08.998925 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:00:08.998925 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:00:08.998925 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:00:08.998925 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:00:08.998925 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:00:08.998925 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:00:08.998925 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:00:08.998925 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:00:08.998925 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:00:08.998925 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:00:08.998925 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 23 15:00:08.998925 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 23 15:00:08.998925 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 23 15:00:08.999862 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 23 15:00:08.999862 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:00:08.999862 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:00:08.999862 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:00:08.999862 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:00:08.999862 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:00:08.999862 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:00:08.999862 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:00:08.999862 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Sep 23 15:00:08.999862 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:00:08.999862 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:00:08.999862 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:00:08.999862 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:00:08.999862 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:00:08.999862 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:00:08.999862 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:00:08.999862 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:00:09.000821 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 23 15:00:09.000821 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 23 15:00:09.000821 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 23 15:00:09.000821 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 23 15:00:09.000821 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 23 15:00:09.000821 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 23 15:00:09.000821 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 15:00:09.000821 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Sep 23 15:00:09.000821 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 507/982] 10.222.0.55 () {56 vars in 1111 bytes} [Fri Sep 23 15:00:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:00:09.007563 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14709db5-b908-4da6-9536-27a92cbfd195 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:00:09.122285 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14709db5-b908-4da6-9536-27a92cbfd195 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:00:09.161335 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a1b916-7adf-4f7f-93dd-9964ad0be46b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:00:09.161945 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a1b916-7adf-4f7f-93dd-9964ad0be46b None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:09.162220 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a1b916-7adf-4f7f-93dd-9964ad0be46b None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:09.162491 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a1b916-7adf-4f7f-93dd-9964ad0be46b None admin] PATH_INFO: `/v3/projects` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:09.163180 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a1b916-7adf-4f7f-93dd-9964ad0be46b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:00:09.163481 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a1b916-7adf-4f7f-93dd-9964ad0be46b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:00:09.164936 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a1b916-7adf-4f7f-93dd-9964ad0be46b None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:00:09.169498 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d596cb-54ca-4c18-93ff-6c80cf93fe0c None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:09.169800 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d596cb-54ca-4c18-93ff-6c80cf93fe0c None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:09.170061 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d596cb-54ca-4c18-93ff-6c80cf93fe0c None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:09.171644 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 508/983] 10.222.0.55 () {54 vars in 927 bytes} [Fri Sep 23 15:00:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:00:09.176758 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee336fa-507c-4099-b12d-2be753ae0010 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:09.177044 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee336fa-507c-4099-b12d-2be753ae0010 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:09.177319 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee336fa-507c-4099-b12d-2be753ae0010 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:09.179317 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 476/984] 10.222.0.55 () {56 vars in 1148 bytes} [Fri Sep 23 15:00:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:00:09.185236 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a1b916-7adf-4f7f-93dd-9964ad0be46b None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:00:09.196903 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:00:09.202649 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-9ee336fa-507c-4099-b12d-2be753ae0010 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:00:09.203991 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-9ee336fa-507c-4099-b12d-2be753ae0010 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:00:09.288127 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 509/985] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Sep 23 15:00:09 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 23 15:00:09.302048 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a1b916-7adf-4f7f-93dd-9964ad0be46b None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:00:09.338695 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aac7964-64e6-48e3-af00-ba47b52e24b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:00:09.339724 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aac7964-64e6-48e3-af00-ba47b52e24b0 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:09.340257 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aac7964-64e6-48e3-af00-ba47b52e24b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:09.340745 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aac7964-64e6-48e3-af00-ba47b52e24b0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:09.341795 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aac7964-64e6-48e3-af00-ba47b52e24b0 None admin] RBAC: Adding query filter params (role.id=f2f7d48ece4f42f682922c1f1beefbd9, scope.project.id=87b3e9d4b6474bd4a04ba59ef52bf3e2, user.id=68206a6b4f244d45bf7bf1504396e430) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:00:09.342318 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aac7964-64e6-48e3-af00-ba47b52e24b0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:00:09.344028 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aac7964-64e6-48e3-af00-ba47b52e24b0 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:00:09.350887 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 477/986] 10.222.0.55 () {56 vars in 1374 bytes} [Fri Sep 23 15:00:09 2022] GET /identity/v3/role_assignments?user.id=68206a6b4f244d45bf7bf1504396e430&scope.project.id=87b3e9d4b6474bd4a04ba59ef52bf3e2&role.id=f2f7d48ece4f42f682922c1f1beefbd9 => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 15:00:09.567602 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee336fa-507c-4099-b12d-2be753ae0010 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:00:09.570057 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:00:09.672821 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee336fa-507c-4099-b12d-2be753ae0010 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:00:09.709815 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f659cd4-0e72-4ab6-bdd1-5f6117855f2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:00:09.710822 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f659cd4-0e72-4ab6-bdd1-5f6117855f2f None admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:09.711319 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f659cd4-0e72-4ab6-bdd1-5f6117855f2f None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:09.711752 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f659cd4-0e72-4ab6-bdd1-5f6117855f2f None admin] PATH_INFO: `/v3/users` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:09.713241 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f659cd4-0e72-4ab6-bdd1-5f6117855f2f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:00:09.714896 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f659cd4-0e72-4ab6-bdd1-5f6117855f2f None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:00:09.770159 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.sql.core [None req-4f659cd4-0e72-4ab6-bdd1-5f6117855f2f None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 23 15:00:09.770159 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 23 15:00:09.770159 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [parameters: {'user_id': '8319f58e5822440f85f1da8d17a1fb9f', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 23 15:00:09.770159 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81994) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 23 15:00:09.778636 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-4f659cd4-0e72-4ab6-bdd1-5f6117855f2f None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 23 15:00:09.778636 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:00:09.778636 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 23 15:00:09.778636 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 23 15:00:09.778636 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 23 15:00:09.778636 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 23 15:00:09.778636 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 23 15:00:09.778636 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application result = self._query(query) Sep 23 15:00:09.778636 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 23 15:00:09.778636 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application conn.query(q) Sep 23 15:00:09.778636 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 23 15:00:09.778636 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 23 15:00:09.778636 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 23 15:00:09.778636 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application result.read() Sep 23 15:00:09.778636 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 23 15:00:09.779702 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 23 15:00:09.779702 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 23 15:00:09.779702 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 23 15:00:09.779702 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 23 15:00:09.779702 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 23 15:00:09.779702 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 23 15:00:09.779702 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 23 15:00:09.779702 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 23 15:00:09.779702 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Sep 23 15:00:09.779702 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 23 15:00:09.779702 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Sep 23 15:00:09.779702 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:00:09.779702 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 23 15:00:09.779702 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 23 15:00:09.779702 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Sep 23 15:00:09.779702 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Sep 23 15:00:09.779702 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 23 15:00:09.779702 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application next(self.gen) Sep 23 15:00:09.780761 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 23 15:00:09.780761 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application yield resource Sep 23 15:00:09.780761 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 23 15:00:09.780761 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application next(self.gen) Sep 23 15:00:09.780761 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 23 15:00:09.780761 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application self.session.rollback() Sep 23 15:00:09.780761 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 23 15:00:09.780761 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application self.force_reraise() Sep 23 15:00:09.780761 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 23 15:00:09.780761 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise self.value Sep 23 15:00:09.780761 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 23 15:00:09.780761 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 23 15:00:09.780761 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 23 15:00:09.780761 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application session.commit() Sep 23 15:00:09.780761 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 23 15:00:09.780761 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 23 15:00:09.780761 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 23 15:00:09.781684 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application self._prepare_impl() Sep 23 15:00:09.781684 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 23 15:00:09.781684 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application self.session.flush() Sep 23 15:00:09.781684 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 23 15:00:09.781684 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application self._flush(objects) Sep 23 15:00:09.781684 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 23 15:00:09.781684 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 23 15:00:09.781684 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 23 15:00:09.781684 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application compat.raise_( Sep 23 15:00:09.781684 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 23 15:00:09.781684 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception Sep 23 15:00:09.781684 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 23 15:00:09.781684 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application flush_context.execute() Sep 23 15:00:09.781684 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 23 15:00:09.781684 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rec.execute(self) Sep 23 15:00:09.781684 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 23 15:00:09.781684 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 23 15:00:09.782614 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 23 15:00:09.782614 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 23 15:00:09.782614 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Sep 23 15:00:09.782614 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application result = connection._execute_20( Sep 23 15:00:09.782614 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 23 15:00:09.782614 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 23 15:00:09.782614 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 23 15:00:09.782614 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 23 15:00:09.782614 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 23 15:00:09.782614 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 23 15:00:09.782614 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 23 15:00:09.782614 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 23 15:00:09.782614 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 23 15:00:09.782614 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 23 15:00:09.782614 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 23 15:00:09.782614 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception Sep 23 15:00:09.783752 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 23 15:00:09.783752 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 23 15:00:09.783752 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 23 15:00:09.783752 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 23 15:00:09.783752 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 23 15:00:09.783752 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application result = self._query(query) Sep 23 15:00:09.783752 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 23 15:00:09.783752 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application conn.query(q) Sep 23 15:00:09.783752 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 23 15:00:09.783752 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 23 15:00:09.783752 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 23 15:00:09.783752 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application result.read() Sep 23 15:00:09.783752 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 23 15:00:09.783752 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 23 15:00:09.783752 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 23 15:00:09.783752 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 23 15:00:09.785227 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 23 15:00:09.785227 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 23 15:00:09.785227 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 23 15:00:09.785227 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 23 15:00:09.785227 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 23 15:00:09.785227 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 23 15:00:09.785227 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [parameters: {'user_id': '8319f58e5822440f85f1da8d17a1fb9f', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 23 15:00:09.785227 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 23 15:00:09.785227 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Sep 23 15:00:09.785227 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 23 15:00:09.785227 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Sep 23 15:00:09.785227 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:00:09.785227 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:00:09.785227 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:00:09.785227 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:00:09.786255 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:00:09.786255 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:00:09.786255 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:00:09.786255 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:00:09.786255 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:00:09.786255 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:00:09.786255 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:00:09.786255 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Sep 23 15:00:09.786255 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Sep 23 15:00:09.786255 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:00:09.786255 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:00:09.786255 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 23 15:00:09.786255 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 15:00:09.786255 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 23 15:00:09.786255 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 15:00:09.786255 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Sep 23 15:00:09.786255 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Sep 23 15:00:09.787254 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Sep 23 15:00:09.787254 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Sep 23 15:00:09.787254 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 23 15:00:09.787254 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 23 15:00:09.787254 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 23 15:00:09.787254 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Sep 23 15:00:09.789346 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 510/987] 10.222.0.55 () {60 vars in 1131 bytes} [Fri Sep 23 15:00:09 2022] POST /identity/v3/users => generated 160 bytes in 222 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:00:09.795619 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aac7964-64e6-48e3-af00-ba47b52e24b0 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:00:09.874604 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:00:09.906711 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aac7964-64e6-48e3-af00-ba47b52e24b0 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:00:09.949962 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74abbd0-ab12-4ced-9fce-8674db5d98ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:00:09.950608 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74abbd0-ab12-4ced-9fce-8674db5d98ed None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:09.950889 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74abbd0-ab12-4ced-9fce-8674db5d98ed None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:09.951163 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74abbd0-ab12-4ced-9fce-8674db5d98ed None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:09.959360 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74abbd0-ab12-4ced-9fce-8674db5d98ed None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:00:09.960954 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74abbd0-ab12-4ced-9fce-8674db5d98ed None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:00:09.973536 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application [None req-e74abbd0-ab12-4ced-9fce-8674db5d98ed None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 23 15:00:09.973536 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:00:09.973536 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:00:09.973536 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:00:09.973536 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:00:09.973536 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:00:09.973536 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:00:09.973536 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:00:09.973536 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:00:09.973536 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:00:09.973536 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:00:09.973536 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:00:09.973536 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 23 15:00:09.973536 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 23 15:00:09.973536 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 23 15:00:09.973536 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 23 15:00:09.974549 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:00:09.974549 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:00:09.974549 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 23 15:00:09.974549 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 15:00:09.974549 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 23 15:00:09.974549 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 15:00:09.974549 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:00:09.974549 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:00:09.974549 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:00:09.974549 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:00:09.974549 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:00:09.974549 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application with Lock( Sep 23 15:00:09.974549 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:00:09.974549 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:00:09.974549 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:00:09.974549 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:00:09.974549 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:00:09.974549 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:00:09.975503 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:00:09.975503 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:00:09.975503 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 23 15:00:09.975503 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 23 15:00:09.975503 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 23 15:00:09.975503 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 23 15:00:09.975503 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 23 15:00:09.975503 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 23 15:00:09.975503 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 23 15:00:09.975503 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Sep 23 15:00:09.975503 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 478/988] 10.222.0.55 () {56 vars in 1099 bytes} [Fri Sep 23 15:00:09 2022] GET /identity/v3/users/cinder => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:00:09.981344 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f659cd4-0e72-4ab6-bdd1-5f6117855f2f None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:00:10.051335 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:00:10.093367 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f659cd4-0e72-4ab6-bdd1-5f6117855f2f None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:00:10.128574 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b32d56-8ac1-43ec-b874-0875c84ce765 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:00:10.129165 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b32d56-8ac1-43ec-b874-0875c84ce765 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:10.129449 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b32d56-8ac1-43ec-b874-0875c84ce765 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:10.129772 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b32d56-8ac1-43ec-b874-0875c84ce765 None admin] PATH_INFO: `/v3/users` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:10.130476 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b32d56-8ac1-43ec-b874-0875c84ce765 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:00:10.130786 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b32d56-8ac1-43ec-b874-0875c84ce765 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:00:10.132558 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b32d56-8ac1-43ec-b874-0875c84ce765 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:00:10.180470 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 511/989] 10.222.0.55 () {56 vars in 1101 bytes} [Fri Sep 23 15:00:09 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:00:10.450663 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930e5783-a4d1-4a68-9580-3f405ba053bf None None] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:10.451136 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930e5783-a4d1-4a68-9580-3f405ba053bf None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:10.451136 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930e5783-a4d1-4a68-9580-3f405ba053bf None None] PATH_INFO: `/` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:10.452737 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 479/990] 10.222.0.55 () {54 vars in 927 bytes} [Fri Sep 23 15:00:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:00:10.458365 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f106f64-dec8-40d8-a5d1-99d407318fdd None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:10.458923 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f106f64-dec8-40d8-a5d1-99d407318fdd None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:10.459396 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f106f64-dec8-40d8-a5d1-99d407318fdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:10.482535 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-6f106f64-dec8-40d8-a5d1-99d407318fdd None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:00:10.484234 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-6f106f64-dec8-40d8-a5d1-99d407318fdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:00:10.554548 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 512/991] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Sep 23 15:00:10 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 23 15:00:10.825212 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930e5783-a4d1-4a68-9580-3f405ba053bf None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:00:10.862888 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfd75e6-cc37-4f5f-8d41-04060f6075b7 None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:10.863451 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfd75e6-cc37-4f5f-8d41-04060f6075b7 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:10.863451 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfd75e6-cc37-4f5f-8d41-04060f6075b7 None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:10.864814 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 513/992] 10.222.0.55 () {54 vars in 927 bytes} [Fri Sep 23 15:00:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 23 15:00:10.869806 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36946bec-5244-4b1e-91c6-943e2802389f None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:10.870086 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36946bec-5244-4b1e-91c6-943e2802389f None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:10.870358 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36946bec-5244-4b1e-91c6-943e2802389f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:10.889646 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-36946bec-5244-4b1e-91c6-943e2802389f None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:00:10.890913 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-36946bec-5244-4b1e-91c6-943e2802389f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:00:10.940195 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930e5783-a4d1-4a68-9580-3f405ba053bf None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:00:10.942090 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:00:10.964918 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 514/993] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Sep 23 15:00:10 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 23 15:00:10.971032 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:00:10.976131 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc43a52d-bbf4-415c-b894-408005abe527 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:00:10.976768 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc43a52d-bbf4-415c-b894-408005abe527 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:10.977047 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc43a52d-bbf4-415c-b894-408005abe527 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:10.977317 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc43a52d-bbf4-415c-b894-408005abe527 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:10.977925 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc43a52d-bbf4-415c-b894-408005abe527 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:00:10.979383 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc43a52d-bbf4-415c-b894-408005abe527 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:00:10.988417 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application [None req-dc43a52d-bbf4-415c-b894-408005abe527 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 23 15:00:10.988417 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:00:10.988417 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 23 15:00:10.988417 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 23 15:00:10.988417 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 23 15:00:10.988417 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 23 15:00:10.988417 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 23 15:00:10.988417 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 23 15:00:10.988417 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 23 15:00:10.988417 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Sep 23 15:00:10.988417 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 23 15:00:10.988417 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Sep 23 15:00:10.988417 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:00:10.988417 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:00:10.988417 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:00:10.988417 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:00:10.988417 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:00:10.994259 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:00:10.994259 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:00:10.994259 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:00:10.994259 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:00:10.994259 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:00:10.994259 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:00:10.994259 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 23 15:00:10.994259 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 23 15:00:10.994259 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 23 15:00:10.994259 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 23 15:00:10.994259 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:00:10.994259 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:00:10.994259 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:00:10.994259 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:00:10.994259 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:00:10.994259 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:00:10.995822 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:00:10.995822 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application with Lock( Sep 23 15:00:10.995822 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:00:10.995822 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:00:10.995822 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:00:10.995822 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:00:10.995822 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:00:10.995822 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:00:10.995822 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:00:10.995822 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:00:10.995822 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 23 15:00:10.995822 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 23 15:00:10.995822 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 23 15:00:10.995822 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Sep 23 15:00:10.995822 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 480/994] 10.222.0.55 () {56 vars in 1126 bytes} [Fri Sep 23 15:00:10 2022] GET /identity/v3/services/object-store => generated 93 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:00:10.997199 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36946bec-5244-4b1e-91c6-943e2802389f None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:00:11.105383 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36946bec-5244-4b1e-91c6-943e2802389f None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:00:11.141291 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf204ac-bf9e-4671-8ec9-3a6066405887 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:00:11.141914 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf204ac-bf9e-4671-8ec9-3a6066405887 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:11.142196 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf204ac-bf9e-4671-8ec9-3a6066405887 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:11.142466 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf204ac-bf9e-4671-8ec9-3a6066405887 None admin] PATH_INFO: `/v3/services` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:11.143095 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf204ac-bf9e-4671-8ec9-3a6066405887 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:00:11.143448 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf204ac-bf9e-4671-8ec9-3a6066405887 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:00:11.144864 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf204ac-bf9e-4671-8ec9-3a6066405887 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:00:11.154588 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 515/995] 10.222.0.55 () {56 vars in 1122 bytes} [Fri Sep 23 15:00:10 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:00:11.156258 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc43a52d-bbf4-415c-b894-408005abe527 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:00:11.238023 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf204ac-bf9e-4671-8ec9-3a6066405887 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:00:11.258496 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc43a52d-bbf4-415c-b894-408005abe527 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:00:11.291769 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:00:11.295118 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e31a66f-c4ce-46bb-9564-12e1d951ca56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:00:11.295735 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e31a66f-c4ce-46bb-9564-12e1d951ca56 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:11.296037 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e31a66f-c4ce-46bb-9564-12e1d951ca56 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:11.296318 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e31a66f-c4ce-46bb-9564-12e1d951ca56 None admin] PATH_INFO: `/v3/services` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:11.296947 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e31a66f-c4ce-46bb-9564-12e1d951ca56 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:00:11.297303 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e31a66f-c4ce-46bb-9564-12e1d951ca56 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:00:11.298673 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e31a66f-c4ce-46bb-9564-12e1d951ca56 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:00:11.303613 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 481/996] 10.222.0.55 () {56 vars in 1122 bytes} [Fri Sep 23 15:00:11 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 15:00:11.310258 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e31a66f-c4ce-46bb-9564-12e1d951ca56 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:00:11.356748 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf204ac-bf9e-4671-8ec9-3a6066405887 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:00:11.371709 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:00:11.396893 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d2ce4a-7b01-466c-92e9-cb177f7be9d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:00:11.397822 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d2ce4a-7b01-466c-92e9-cb177f7be9d3 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:11.398110 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d2ce4a-7b01-466c-92e9-cb177f7be9d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:11.398445 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d2ce4a-7b01-466c-92e9-cb177f7be9d3 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:11.399036 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d2ce4a-7b01-466c-92e9-cb177f7be9d3 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:00:11.400892 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d2ce4a-7b01-466c-92e9-cb177f7be9d3 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:00:11.411162 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-f0d2ce4a-7b01-466c-92e9-cb177f7be9d3 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 23 15:00:11.411162 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:00:11.411162 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 23 15:00:11.411162 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 23 15:00:11.411162 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 23 15:00:11.411162 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 23 15:00:11.411162 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 23 15:00:11.411162 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 23 15:00:11.411162 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 23 15:00:11.411162 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Sep 23 15:00:11.411162 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 23 15:00:11.411162 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Sep 23 15:00:11.411162 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:00:11.411162 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:00:11.411162 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:00:11.411162 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:00:11.411162 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:00:11.412219 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:00:11.412219 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:00:11.412219 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:00:11.412219 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:00:11.412219 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:00:11.412219 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:00:11.412219 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 23 15:00:11.412219 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 23 15:00:11.412219 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 23 15:00:11.412219 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 23 15:00:11.412219 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:00:11.412219 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:00:11.412219 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:00:11.412219 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:00:11.412219 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:00:11.412219 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:00:11.413070 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:00:11.413070 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Sep 23 15:00:11.413070 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:00:11.413070 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:00:11.413070 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:00:11.413070 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:00:11.413070 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:00:11.413070 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:00:11.413070 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:00:11.413070 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:00:11.413070 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 23 15:00:11.413070 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 23 15:00:11.413070 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 23 15:00:11.413070 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Sep 23 15:00:11.414574 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e31a66f-c4ce-46bb-9564-12e1d951ca56 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:00:11.416313 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 516/997] 10.222.0.55 () {56 vars in 1111 bytes} [Fri Sep 23 15:00:11 2022] GET /identity/v3/services/compute => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 23 15:00:11.419281 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d2ce4a-7b01-466c-92e9-cb177f7be9d3 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:00:11.454143 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d692e8-5391-45f1-a2de-518cb2089412 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:00:11.454848 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d692e8-5391-45f1-a2de-518cb2089412 None admin] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:11.455040 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d692e8-5391-45f1-a2de-518cb2089412 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:11.455296 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d692e8-5391-45f1-a2de-518cb2089412 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:11.455954 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d692e8-5391-45f1-a2de-518cb2089412 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:00:11.457374 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d692e8-5391-45f1-a2de-518cb2089412 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:00:11.521432 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d2ce4a-7b01-466c-92e9-cb177f7be9d3 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:00:11.531648 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:00:11.557440 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cc39f6-7fa5-4e23-ab8b-bdb62a32a655 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:00:11.558094 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cc39f6-7fa5-4e23-ab8b-bdb62a32a655 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:11.558375 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cc39f6-7fa5-4e23-ab8b-bdb62a32a655 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:11.558646 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cc39f6-7fa5-4e23-ab8b-bdb62a32a655 None admin] PATH_INFO: `/v3/services` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:11.559262 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cc39f6-7fa5-4e23-ab8b-bdb62a32a655 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:00:11.559562 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cc39f6-7fa5-4e23-ab8b-bdb62a32a655 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:00:11.560938 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cc39f6-7fa5-4e23-ab8b-bdb62a32a655 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:00:11.565650 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 517/998] 10.222.0.55 () {56 vars in 1112 bytes} [Fri Sep 23 15:00:11 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:00:11.571190 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cc39f6-7fa5-4e23-ab8b-bdb62a32a655 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:00:11.619961 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 482/999] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Sep 23 15:00:11 2022] POST /identity/v3/endpoints => generated 325 bytes in 310 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:00:11.693683 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cc39f6-7fa5-4e23-ab8b-bdb62a32a655 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:00:11.704474 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fa5f7b-1658-4c31-9116-b57e4f3fa574 None None] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:11.704763 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fa5f7b-1658-4c31-9116-b57e4f3fa574 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:11.705033 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fa5f7b-1658-4c31-9116-b57e4f3fa574 None None] PATH_INFO: `/` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:11.706384 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 483/1000] 10.222.0.55 () {54 vars in 927 bytes} [Fri Sep 23 15:00:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 23 15:00:11.711965 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730fc94c-291d-48ba-878e-f03de2834e11 None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:11.716055 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730fc94c-291d-48ba-878e-f03de2834e11 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:11.716055 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730fc94c-291d-48ba-878e-f03de2834e11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:11.733058 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-730fc94c-291d-48ba-878e-f03de2834e11 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:00:11.734280 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-730fc94c-291d-48ba-878e-f03de2834e11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:00:11.745751 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c55f255-787e-499b-bd78-64df71bebd90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:00:11.746386 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c55f255-787e-499b-bd78-64df71bebd90 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:11.747281 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c55f255-787e-499b-bd78-64df71bebd90 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:11.747281 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c55f255-787e-499b-bd78-64df71bebd90 None admin] PATH_INFO: `/v3/services` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:11.747944 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c55f255-787e-499b-bd78-64df71bebd90 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:00:11.748351 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c55f255-787e-499b-bd78-64df71bebd90 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:00:11.749979 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c55f255-787e-499b-bd78-64df71bebd90 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:00:11.761032 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 518/1001] 10.222.0.55 () {56 vars in 1112 bytes} [Fri Sep 23 15:00:11 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 15:00:11.762244 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c55f255-787e-499b-bd78-64df71bebd90 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:00:11.813078 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:00:11.815708 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 484/1002] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Sep 23 15:00:11 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 23 15:00:11.883902 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c55f255-787e-499b-bd78-64df71bebd90 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:00:11.919451 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a3b834-f3c9-4a29-9883-5d6d6d2df460 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:00:11.920357 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a3b834-f3c9-4a29-9883-5d6d6d2df460 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:11.920786 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a3b834-f3c9-4a29-9883-5d6d6d2df460 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:11.921203 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a3b834-f3c9-4a29-9883-5d6d6d2df460 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:11.922044 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a3b834-f3c9-4a29-9883-5d6d6d2df460 None admin] RBAC: Adding query filter params (interface=public, service_id=a1150f3b6cce4db9a29529f374cb2d9f, region_id=RegionOne) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:00:11.922515 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a3b834-f3c9-4a29-9883-5d6d6d2df460 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:00:11.924182 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a3b834-f3c9-4a29-9883-5d6d6d2df460 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:00:11.929811 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 519/1003] 10.222.0.55 () {56 vars in 1251 bytes} [Fri Sep 23 15:00:11 2022] GET /identity/v3/endpoints?service_id=a1150f3b6cce4db9a29529f374cb2d9f&interface=public®ion_id=RegionOne => generated 194 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 15:00:11.935303 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730fc94c-291d-48ba-878e-f03de2834e11 None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:00:11.986604 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:00:12.037306 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730fc94c-291d-48ba-878e-f03de2834e11 None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:00:12.080053 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802fea32-cada-4de5-a24a-29a729e35283 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:00:12.080678 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802fea32-cada-4de5-a24a-29a729e35283 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:12.080958 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802fea32-cada-4de5-a24a-29a729e35283 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:12.081233 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802fea32-cada-4de5-a24a-29a729e35283 None admin] PATH_INFO: `/v3/services` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:12.081843 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802fea32-cada-4de5-a24a-29a729e35283 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:00:12.083282 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802fea32-cada-4de5-a24a-29a729e35283 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:00:12.089281 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a3b834-f3c9-4a29-9883-5d6d6d2df460 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:00:12.095368 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 485/1004] 10.222.0.55 () {56 vars in 1087 bytes} [Fri Sep 23 15:00:11 2022] GET /identity/v3/services => generated 1727 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:00:12.200468 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:00:12.204940 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a3b834-f3c9-4a29-9883-5d6d6d2df460 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:00:12.241785 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82a64a8-94d6-462f-86f3-6791051b0065 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:00:12.242430 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82a64a8-94d6-462f-86f3-6791051b0065 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:12.242707 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82a64a8-94d6-462f-86f3-6791051b0065 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:12.242975 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82a64a8-94d6-462f-86f3-6791051b0065 None admin] PATH_INFO: `/v3/services/image` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:12.243649 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82a64a8-94d6-462f-86f3-6791051b0065 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:00:12.245342 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82a64a8-94d6-462f-86f3-6791051b0065 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:00:12.255072 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-d82a64a8-94d6-462f-86f3-6791051b0065 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 23 15:00:12.255072 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:00:12.255072 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 23 15:00:12.255072 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 23 15:00:12.255072 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 23 15:00:12.255072 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 23 15:00:12.255072 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 23 15:00:12.255072 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 23 15:00:12.255072 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 23 15:00:12.255072 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Sep 23 15:00:12.255072 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 23 15:00:12.255072 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Sep 23 15:00:12.255072 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:00:12.255072 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:00:12.255072 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:00:12.255072 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:00:12.255072 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:00:12.256499 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:00:12.256499 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:00:12.256499 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:00:12.256499 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:00:12.256499 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:00:12.256499 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:00:12.256499 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 23 15:00:12.256499 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 23 15:00:12.256499 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 23 15:00:12.256499 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 23 15:00:12.256499 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:00:12.256499 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:00:12.256499 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:00:12.256499 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:00:12.256499 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:00:12.256499 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:00:12.257649 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:00:12.257649 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Sep 23 15:00:12.257649 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:00:12.257649 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:00:12.257649 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:00:12.257649 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:00:12.257649 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:00:12.257649 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:00:12.257649 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:00:12.257649 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:00:12.257649 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 23 15:00:12.257649 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 23 15:00:12.257649 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 23 15:00:12.257649 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Sep 23 15:00:12.260422 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 520/1005] 10.222.0.55 () {56 vars in 1105 bytes} [Fri Sep 23 15:00:12 2022] GET /identity/v3/services/image => generated 86 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 23 15:00:12.262914 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802fea32-cada-4de5-a24a-29a729e35283 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:00:12.370132 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802fea32-cada-4de5-a24a-29a729e35283 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:00:12.385055 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:00:12.408702 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ab669d-25e5-4ea0-925e-7720c252537e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:00:12.409306 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ab669d-25e5-4ea0-925e-7720c252537e None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:12.409659 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ab669d-25e5-4ea0-925e-7720c252537e None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:12.409926 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ab669d-25e5-4ea0-925e-7720c252537e None admin] PATH_INFO: `/v3/services` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:12.410562 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ab669d-25e5-4ea0-925e-7720c252537e None admin] RBAC: Adding query filter params (name=image) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:00:12.410859 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ab669d-25e5-4ea0-925e-7720c252537e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:00:12.412242 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ab669d-25e5-4ea0-925e-7720c252537e None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:00:12.417288 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 486/1006] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Sep 23 15:00:12 2022] GET /identity/v3/services?name=image => generated 122 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:00:12.423112 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82a64a8-94d6-462f-86f3-6791051b0065 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:00:12.529912 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82a64a8-94d6-462f-86f3-6791051b0065 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:00:12.566831 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f01165d-4d2c-464a-9277-cd8d29901e07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:00:12.567875 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f01165d-4d2c-464a-9277-cd8d29901e07 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:12.568393 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f01165d-4d2c-464a-9277-cd8d29901e07 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:12.568889 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f01165d-4d2c-464a-9277-cd8d29901e07 None admin] PATH_INFO: `/v3/services` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:12.569726 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f01165d-4d2c-464a-9277-cd8d29901e07 None admin] RBAC: Adding query filter params (type=image) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:00:12.570246 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f01165d-4d2c-464a-9277-cd8d29901e07 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:00:12.572783 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f01165d-4d2c-464a-9277-cd8d29901e07 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:00:12.578357 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 521/1007] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Sep 23 15:00:12 2022] GET /identity/v3/services?type=image => generated 122 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 15:00:13.868293 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b64708-34d8-40e6-871f-9bead4e5a290 None None] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:13.868824 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b64708-34d8-40e6-871f-9bead4e5a290 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:13.868824 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b64708-34d8-40e6-871f-9bead4e5a290 None None] PATH_INFO: `/` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:13.871455 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 487/1008] 10.222.0.55 () {54 vars in 927 bytes} [Fri Sep 23 15:00:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:00:13.876025 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2436a92c-75d9-460b-9b84-71e7346bf9e9 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:13.876477 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2436a92c-75d9-460b-9b84-71e7346bf9e9 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:13.876652 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2436a92c-75d9-460b-9b84-71e7346bf9e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:13.900448 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-2436a92c-75d9-460b-9b84-71e7346bf9e9 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:00:13.901771 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-2436a92c-75d9-460b-9b84-71e7346bf9e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:00:13.924430 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:00:13.973745 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 522/1009] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Sep 23 15:00:13 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 23 15:00:14.250238 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b64708-34d8-40e6-871f-9bead4e5a290 None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:00:14.313793 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee665b3-2309-466d-b0a1-258e06ca8d9e None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:14.314275 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee665b3-2309-466d-b0a1-258e06ca8d9e None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:14.314555 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee665b3-2309-466d-b0a1-258e06ca8d9e None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:14.316022 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 523/1010] 10.222.0.55 () {54 vars in 927 bytes} [Fri Sep 23 15:00:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 23 15:00:14.322268 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f87746-04f7-4747-8d6a-966f2bd46985 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:14.322555 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f87746-04f7-4747-8d6a-966f2bd46985 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:14.322981 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f87746-04f7-4747-8d6a-966f2bd46985 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:14.342088 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-37f87746-04f7-4747-8d6a-966f2bd46985 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:00:14.343279 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-37f87746-04f7-4747-8d6a-966f2bd46985 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:00:14.362603 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b64708-34d8-40e6-871f-9bead4e5a290 None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:00:14.397716 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c87fe7-5999-4fde-a67c-15c7d28c133a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:00:14.398347 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c87fe7-5999-4fde-a67c-15c7d28c133a None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:14.398625 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c87fe7-5999-4fde-a67c-15c7d28c133a None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:14.398894 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c87fe7-5999-4fde-a67c-15c7d28c133a None admin] PATH_INFO: `/v3/domains/default` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:14.400061 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c87fe7-5999-4fde-a67c-15c7d28c133a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:00:14.401487 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c87fe7-5999-4fde-a67c-15c7d28c133a None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:00:14.403048 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 488/1011] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Sep 23 15:00:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:00:14.408839 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c87fe7-5999-4fde-a67c-15c7d28c133a None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:00:14.418758 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 524/1012] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Sep 23 15:00:14 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 23 15:00:14.465260 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:00:14.518770 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c87fe7-5999-4fde-a67c-15c7d28c133a None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:00:14.553903 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e704a37-c200-476c-b5f6-de8bd2aed951 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:00:14.554511 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e704a37-c200-476c-b5f6-de8bd2aed951 None admin] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:14.554786 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e704a37-c200-476c-b5f6-de8bd2aed951 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:14.555056 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e704a37-c200-476c-b5f6-de8bd2aed951 None admin] PATH_INFO: `/v3/projects` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:14.555700 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e704a37-c200-476c-b5f6-de8bd2aed951 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:00:14.557257 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e704a37-c200-476c-b5f6-de8bd2aed951 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:00:14.699825 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 489/1013] 10.222.0.55 () {60 vars in 1141 bytes} [Fri Sep 23 15:00:14 2022] POST /identity/v3/projects => generated 311 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 23 15:00:14.700059 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f87746-04f7-4747-8d6a-966f2bd46985 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:00:14.753674 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05befee-46c5-4357-b8cf-57e1a1ce81d6 None None] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:14.753966 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05befee-46c5-4357-b8cf-57e1a1ce81d6 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:14.754240 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05befee-46c5-4357-b8cf-57e1a1ce81d6 None None] PATH_INFO: `/` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:14.755667 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 490/1014] 10.222.0.55 () {54 vars in 927 bytes} [Fri Sep 23 15:00:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 23 15:00:14.762194 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b485120-3488-45eb-b722-083b9de14972 None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:14.762490 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b485120-3488-45eb-b722-083b9de14972 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:14.762766 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b485120-3488-45eb-b722-083b9de14972 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:14.782107 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-5b485120-3488-45eb-b722-083b9de14972 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:00:14.783511 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-5b485120-3488-45eb-b722-083b9de14972 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:00:14.805013 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:00:14.824194 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f87746-04f7-4747-8d6a-966f2bd46985 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:00:14.862189 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16ac5fc-f7bd-4b3c-a230-8614c88fdd72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:00:14.862842 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16ac5fc-f7bd-4b3c-a230-8614c88fdd72 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:14.863087 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 491/1015] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Sep 23 15:00:14 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 23 15:00:14.863196 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16ac5fc-f7bd-4b3c-a230-8614c88fdd72 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:14.863388 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16ac5fc-f7bd-4b3c-a230-8614c88fdd72 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:14.864007 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16ac5fc-f7bd-4b3c-a230-8614c88fdd72 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:00:14.865392 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16ac5fc-f7bd-4b3c-a230-8614c88fdd72 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:00:14.876660 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-d16ac5fc-f7bd-4b3c-a230-8614c88fdd72 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 23 15:00:14.876660 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:00:14.876660 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 23 15:00:14.876660 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 23 15:00:14.876660 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 23 15:00:14.876660 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 23 15:00:14.876660 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 23 15:00:14.876660 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 23 15:00:14.876660 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 23 15:00:14.876660 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Sep 23 15:00:14.876660 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 23 15:00:14.876660 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Sep 23 15:00:14.876660 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:00:14.876660 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:00:14.876660 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:00:14.876660 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:00:14.876660 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:00:14.878311 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:00:14.878311 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:00:14.878311 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:00:14.878311 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:00:14.878311 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:00:14.878311 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:00:14.878311 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 23 15:00:14.878311 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 23 15:00:14.878311 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 23 15:00:14.878311 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 23 15:00:14.878311 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:00:14.878311 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:00:14.878311 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:00:14.878311 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:00:14.878311 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:00:14.878311 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:00:14.879392 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:00:14.879392 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Sep 23 15:00:14.879392 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:00:14.879392 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:00:14.879392 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:00:14.879392 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:00:14.879392 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:00:14.879392 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:00:14.879392 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:00:14.879392 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:00:14.879392 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 23 15:00:14.879392 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 23 15:00:14.879392 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 23 15:00:14.879392 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Sep 23 15:00:14.882969 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 525/1016] 10.222.0.55 () {56 vars in 1111 bytes} [Fri Sep 23 15:00:14 2022] GET /identity/v3/services/compute => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:00:14.884663 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b485120-3488-45eb-b722-083b9de14972 None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:00:14.919453 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:00:14.995494 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b485120-3488-45eb-b722-083b9de14972 None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:00:15.034065 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14ecf4d-cc67-4d36-b440-3b0dd8621eb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:00:15.034605 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14ecf4d-cc67-4d36-b440-3b0dd8621eb0 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:15.034886 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14ecf4d-cc67-4d36-b440-3b0dd8621eb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:15.035159 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14ecf4d-cc67-4d36-b440-3b0dd8621eb0 None admin] PATH_INFO: `/v3/services` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:15.035882 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14ecf4d-cc67-4d36-b440-3b0dd8621eb0 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:00:15.036214 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14ecf4d-cc67-4d36-b440-3b0dd8621eb0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:00:15.037618 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14ecf4d-cc67-4d36-b440-3b0dd8621eb0 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:00:15.043329 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 492/1017] 10.222.0.55 () {56 vars in 1112 bytes} [Fri Sep 23 15:00:14 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:00:15.048923 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16ac5fc-f7bd-4b3c-a230-8614c88fdd72 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:00:15.142448 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14ecf4d-cc67-4d36-b440-3b0dd8621eb0 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:00:15.166332 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16ac5fc-f7bd-4b3c-a230-8614c88fdd72 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:00:15.205829 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef82e014-760a-4db5-96e4-d431c963d9c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:00:15.206443 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef82e014-760a-4db5-96e4-d431c963d9c7 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:15.206716 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef82e014-760a-4db5-96e4-d431c963d9c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:15.206990 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef82e014-760a-4db5-96e4-d431c963d9c7 None admin] PATH_INFO: `/v3/services` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:15.207620 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef82e014-760a-4db5-96e4-d431c963d9c7 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:00:15.207920 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef82e014-760a-4db5-96e4-d431c963d9c7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:00:15.209299 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef82e014-760a-4db5-96e4-d431c963d9c7 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:00:15.219153 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 526/1018] 10.222.0.55 () {56 vars in 1112 bytes} [Fri Sep 23 15:00:15 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:00:15.220226 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef82e014-760a-4db5-96e4-d431c963d9c7 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:00:15.252433 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:00:15.256826 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:00:15.266949 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14ecf4d-cc67-4d36-b440-3b0dd8621eb0 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:00:15.302810 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679047c7-793d-4738-b0f2-b46908baf97e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:00:15.303340 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679047c7-793d-4738-b0f2-b46908baf97e None admin] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:15.303616 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679047c7-793d-4738-b0f2-b46908baf97e None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:15.303883 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679047c7-793d-4738-b0f2-b46908baf97e None admin] PATH_INFO: `/v3/services` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:15.304566 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679047c7-793d-4738-b0f2-b46908baf97e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:00:15.306525 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679047c7-793d-4738-b0f2-b46908baf97e None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:00:15.343382 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef82e014-760a-4db5-96e4-d431c963d9c7 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:00:15.378026 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54d38ac-e544-4709-ba50-0a46ca283bfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:00:15.378679 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54d38ac-e544-4709-ba50-0a46ca283bfc None admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:15.378959 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54d38ac-e544-4709-ba50-0a46ca283bfc None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:15.379233 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54d38ac-e544-4709-ba50-0a46ca283bfc None admin] PATH_INFO: `/v3/endpoints` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:15.379897 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54d38ac-e544-4709-ba50-0a46ca283bfc None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:00:15.381291 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54d38ac-e544-4709-ba50-0a46ca283bfc None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:00:15.468443 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 493/1019] 10.222.0.55 () {60 vars in 1141 bytes} [Fri Sep 23 15:00:15 2022] POST /identity/v3/services => generated 242 bytes in 326 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 23 15:00:15.511472 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 527/1020] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Sep 23 15:00:15 2022] POST /identity/v3/endpoints => generated 334 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:00:16.938918 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1313b1c2-07d4-4f70-a64e-8ec9ff3524f1 None None] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:16.939474 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1313b1c2-07d4-4f70-a64e-8ec9ff3524f1 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:16.939474 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1313b1c2-07d4-4f70-a64e-8ec9ff3524f1 None None] PATH_INFO: `/` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:16.940835 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 494/1021] 10.222.0.55 () {54 vars in 927 bytes} [Fri Sep 23 15:00:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:00:16.945960 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e36b441-1208-439a-a8c4-36b8082c1b90 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:16.946342 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e36b441-1208-439a-a8c4-36b8082c1b90 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:16.947117 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e36b441-1208-439a-a8c4-36b8082c1b90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:16.966249 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-6e36b441-1208-439a-a8c4-36b8082c1b90 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:00:16.967431 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-6e36b441-1208-439a-a8c4-36b8082c1b90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:00:17.031710 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:00:17.043211 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 528/1022] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Sep 23 15:00:16 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 23 15:00:17.310700 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1313b1c2-07d4-4f70-a64e-8ec9ff3524f1 None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:00:17.420612 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1313b1c2-07d4-4f70-a64e-8ec9ff3524f1 None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:00:17.455271 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39742ee0-8f21-4a78-a7a4-1bfe633cfbff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:00:17.455909 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39742ee0-8f21-4a78-a7a4-1bfe633cfbff None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:17.456214 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39742ee0-8f21-4a78-a7a4-1bfe633cfbff None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:17.456485 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39742ee0-8f21-4a78-a7a4-1bfe633cfbff None admin] PATH_INFO: `/v3/domains/default` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:17.457609 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39742ee0-8f21-4a78-a7a4-1bfe633cfbff None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:00:17.459048 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39742ee0-8f21-4a78-a7a4-1bfe633cfbff None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:00:17.460624 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 495/1023] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Sep 23 15:00:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:00:17.466707 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e36b441-1208-439a-a8c4-36b8082c1b90 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:00:17.571889 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e36b441-1208-439a-a8c4-36b8082c1b90 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:00:17.609470 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf806be-f1d9-4a0c-9000-61ccf500e650 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:00:17.610091 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf806be-f1d9-4a0c-9000-61ccf500e650 None admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:17.610353 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf806be-f1d9-4a0c-9000-61ccf500e650 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:17.610625 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf806be-f1d9-4a0c-9000-61ccf500e650 None admin] PATH_INFO: `/v3/users` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:17.611273 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf806be-f1d9-4a0c-9000-61ccf500e650 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:00:17.612715 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf806be-f1d9-4a0c-9000-61ccf500e650 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:00:17.688697 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a24bbc-c2c8-4019-aec2-7a7702bef2e6 None None] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:17.689080 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a24bbc-c2c8-4019-aec2-7a7702bef2e6 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:17.689425 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a24bbc-c2c8-4019-aec2-7a7702bef2e6 None None] PATH_INFO: `/` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:17.690895 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 496/1024] 10.222.0.55 () {54 vars in 927 bytes} [Fri Sep 23 15:00:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 23 15:00:17.695831 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269193c6-00b8-4e6e-82c8-9246fca3af22 None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:17.696263 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269193c6-00b8-4e6e-82c8-9246fca3af22 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:17.696624 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269193c6-00b8-4e6e-82c8-9246fca3af22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:17.719027 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-269193c6-00b8-4e6e-82c8-9246fca3af22 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:00:17.720310 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-269193c6-00b8-4e6e-82c8-9246fca3af22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:00:17.733938 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 529/1025] 10.222.0.55 () {60 vars in 1132 bytes} [Fri Sep 23 15:00:17 2022] POST /identity/v3/users => generated 285 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 23 15:00:17.745486 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:00:17.808571 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 497/1026] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Sep 23 15:00:17 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 23 15:00:17.838658 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016d3f62-23f5-4868-9b6d-a7aa88b9f69c None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:17.838854 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016d3f62-23f5-4868-9b6d-a7aa88b9f69c None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:17.839111 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016d3f62-23f5-4868-9b6d-a7aa88b9f69c None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:17.840708 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 530/1027] 10.222.0.55 () {54 vars in 927 bytes} [Fri Sep 23 15:00:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:00:17.846270 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff299097-31ba-43ed-9a7d-fada8429c08a None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:17.847082 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff299097-31ba-43ed-9a7d-fada8429c08a None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:17.847747 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff299097-31ba-43ed-9a7d-fada8429c08a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:17.868798 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-ff299097-31ba-43ed-9a7d-fada8429c08a None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:00:17.870185 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-ff299097-31ba-43ed-9a7d-fada8429c08a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:00:17.947951 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 498/1028] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Sep 23 15:00:17 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 23 15:00:18.101594 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016d3f62-23f5-4868-9b6d-a7aa88b9f69c None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:00:18.160496 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:00:18.209797 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016d3f62-23f5-4868-9b6d-a7aa88b9f69c None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:00:18.221223 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff299097-31ba-43ed-9a7d-fada8429c08a None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:00:18.246636 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c710e6e-dc10-4e00-b3b9-561ed8ff1845 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:00:18.247336 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c710e6e-dc10-4e00-b3b9-561ed8ff1845 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:18.247618 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c710e6e-dc10-4e00-b3b9-561ed8ff1845 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:18.247888 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c710e6e-dc10-4e00-b3b9-561ed8ff1845 None admin] PATH_INFO: `/v3/services/image` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:18.248507 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c710e6e-dc10-4e00-b3b9-561ed8ff1845 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:00:18.250050 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c710e6e-dc10-4e00-b3b9-561ed8ff1845 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:00:18.259873 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-5c710e6e-dc10-4e00-b3b9-561ed8ff1845 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 23 15:00:18.259873 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:00:18.259873 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 23 15:00:18.259873 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 23 15:00:18.259873 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 23 15:00:18.259873 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 23 15:00:18.259873 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 23 15:00:18.259873 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 23 15:00:18.259873 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 23 15:00:18.259873 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Sep 23 15:00:18.259873 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 23 15:00:18.259873 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Sep 23 15:00:18.259873 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:00:18.259873 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:00:18.259873 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:00:18.259873 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:00:18.259873 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:00:18.264295 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:00:18.264295 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:00:18.264295 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:00:18.264295 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:00:18.264295 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:00:18.264295 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:00:18.264295 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 23 15:00:18.264295 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 23 15:00:18.264295 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 23 15:00:18.264295 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 23 15:00:18.264295 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:00:18.264295 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:00:18.264295 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:00:18.264295 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:00:18.264295 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:00:18.264295 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:00:18.265204 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:00:18.265204 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Sep 23 15:00:18.265204 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:00:18.265204 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:00:18.265204 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:00:18.265204 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:00:18.265204 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:00:18.265204 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:00:18.265204 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:00:18.265204 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:00:18.265204 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 23 15:00:18.265204 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 23 15:00:18.265204 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 23 15:00:18.265204 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Sep 23 15:00:18.265204 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 531/1029] 10.222.0.55 () {56 vars in 1105 bytes} [Fri Sep 23 15:00:18 2022] GET /identity/v3/services/image => generated 86 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:00:18.267587 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c710e6e-dc10-4e00-b3b9-561ed8ff1845 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:00:18.312101 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:00:18.333953 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff299097-31ba-43ed-9a7d-fada8429c08a None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:00:18.369315 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf5b43d-0b5c-4cfa-a1aa-62cfaae1d9f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:00:18.369872 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf5b43d-0b5c-4cfa-a1aa-62cfaae1d9f6 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:18.370152 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf5b43d-0b5c-4cfa-a1aa-62cfaae1d9f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:18.370427 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf5b43d-0b5c-4cfa-a1aa-62cfaae1d9f6 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:18.378371 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c710e6e-dc10-4e00-b3b9-561ed8ff1845 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:00:18.380665 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf5b43d-0b5c-4cfa-a1aa-62cfaae1d9f6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 15:00:18.385989 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application [None req-2cf5b43d-0b5c-4cfa-a1aa-62cfaae1d9f6 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 23 15:00:18.385989 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:00:18.385989 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:00:18.385989 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:00:18.385989 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:00:18.385989 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:00:18.385989 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:00:18.385989 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:00:18.385989 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:00:18.385989 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:00:18.385989 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:00:18.385989 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:00:18.385989 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 15:00:18.385989 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 15:00:18.385989 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 15:00:18.385989 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 15:00:18.386938 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 15:00:18.386938 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 15:00:18.386938 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 15:00:18.386938 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 15:00:18.386938 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:00:18.386938 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:00:18.386938 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:00:18.386938 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:00:18.386938 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:00:18.386938 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:00:18.386938 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:00:18.386938 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application with Lock( Sep 23 15:00:18.386938 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:00:18.386938 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:00:18.386938 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:00:18.386938 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:00:18.388470 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:00:18.388470 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:00:18.388470 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:00:18.388470 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:00:18.388470 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 15:00:18.388470 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 15:00:18.388470 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 15:00:18.388470 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 15:00:18.388470 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 15:00:18.388470 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 15:00:18.388470 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 23 15:00:18.388470 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Sep 23 15:00:18.388470 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 499/1030] 10.222.0.55 () {56 vars in 1120 bytes} [Fri Sep 23 15:00:18 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 23 15:00:18.392932 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf5b43d-0b5c-4cfa-a1aa-62cfaae1d9f6 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:00:18.399604 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:00:18.418593 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c965685f-9183-4510-8f18-57cb019c0baa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:00:18.419204 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c965685f-9183-4510-8f18-57cb019c0baa None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:18.419534 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c965685f-9183-4510-8f18-57cb019c0baa None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:18.419798 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c965685f-9183-4510-8f18-57cb019c0baa None admin] PATH_INFO: `/v3/services` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:18.420410 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c965685f-9183-4510-8f18-57cb019c0baa None admin] RBAC: Adding query filter params (name=image) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:00:18.420714 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c965685f-9183-4510-8f18-57cb019c0baa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:00:18.422075 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c965685f-9183-4510-8f18-57cb019c0baa None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:00:18.427022 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 532/1031] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Sep 23 15:00:18 2022] GET /identity/v3/services?name=image => generated 122 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:00:18.432598 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c965685f-9183-4510-8f18-57cb019c0baa None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:00:18.506710 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf5b43d-0b5c-4cfa-a1aa-62cfaae1d9f6 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:00:18.539761 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c965685f-9183-4510-8f18-57cb019c0baa None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:00:18.543439 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69daddd-9457-4e7f-8d8b-43f54c3ae24c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:00:18.544083 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69daddd-9457-4e7f-8d8b-43f54c3ae24c None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:18.544360 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69daddd-9457-4e7f-8d8b-43f54c3ae24c None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:18.544631 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69daddd-9457-4e7f-8d8b-43f54c3ae24c None admin] PATH_INFO: `/v3/roles` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:18.545286 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69daddd-9457-4e7f-8d8b-43f54c3ae24c None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:00:18.545591 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69daddd-9457-4e7f-8d8b-43f54c3ae24c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:00:18.546952 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69daddd-9457-4e7f-8d8b-43f54c3ae24c None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:00:18.555186 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 500/1032] 10.222.0.55 () {56 vars in 1115 bytes} [Fri Sep 23 15:00:18 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 15:00:18.561740 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69daddd-9457-4e7f-8d8b-43f54c3ae24c None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:00:18.564528 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:00:18.582774 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548f0ebc-75f7-4061-a8d0-ab23e9f616af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:00:18.583462 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548f0ebc-75f7-4061-a8d0-ab23e9f616af None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:18.583746 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548f0ebc-75f7-4061-a8d0-ab23e9f616af None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:18.584044 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548f0ebc-75f7-4061-a8d0-ab23e9f616af None admin] PATH_INFO: `/v3/services` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:18.584676 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548f0ebc-75f7-4061-a8d0-ab23e9f616af None admin] RBAC: Adding query filter params (type=image) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:00:18.584977 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548f0ebc-75f7-4061-a8d0-ab23e9f616af None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:00:18.586325 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548f0ebc-75f7-4061-a8d0-ab23e9f616af None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:00:18.592436 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 533/1033] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Sep 23 15:00:18 2022] GET /identity/v3/services?type=image => generated 350 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:00:18.598758 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548f0ebc-75f7-4061-a8d0-ab23e9f616af None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:00:18.648444 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:00:18.672122 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69daddd-9457-4e7f-8d8b-43f54c3ae24c None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:00:18.709210 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757744f5-0aca-40fc-a3fe-d463dc9728a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:00:18.709872 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757744f5-0aca-40fc-a3fe-d463dc9728a1 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:18.710151 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757744f5-0aca-40fc-a3fe-d463dc9728a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:18.710427 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757744f5-0aca-40fc-a3fe-d463dc9728a1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:18.710615 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548f0ebc-75f7-4061-a8d0-ab23e9f616af None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:00:18.724815 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757744f5-0aca-40fc-a3fe-d463dc9728a1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:00:18.726641 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757744f5-0aca-40fc-a3fe-d463dc9728a1 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:00:18.742283 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application [None req-757744f5-0aca-40fc-a3fe-d463dc9728a1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 15:00:18.742283 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:00:18.742283 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:00:18.742283 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:00:18.742283 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:00:18.742283 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:00:18.742283 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:00:18.742283 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:00:18.742283 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:00:18.742283 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:00:18.742283 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:00:18.742283 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:00:18.742283 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 15:00:18.742283 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 15:00:18.742283 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 15:00:18.742283 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 15:00:18.769965 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:00:18.769965 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:00:18.769965 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:00:18.769965 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:00:18.769965 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:00:18.769965 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:00:18.769965 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:00:18.769965 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application with Lock( Sep 23 15:00:18.769965 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:00:18.769965 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:00:18.769965 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:00:18.769965 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:00:18.769965 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:00:18.769965 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:00:18.769965 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:00:18.769965 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:00:18.769965 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 15:00:18.770894 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 15:00:18.770894 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 15:00:18.770894 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Sep 23 15:00:18.770894 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 501/1034] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Sep 23 15:00:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 23 15:00:18.770894 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757744f5-0aca-40fc-a3fe-d463dc9728a1 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:00:18.771287 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4962bf-131a-40c1-af60-21bf6f37513e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:00:18.771287 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4962bf-131a-40c1-af60-21bf6f37513e None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:18.771287 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4962bf-131a-40c1-af60-21bf6f37513e None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:18.771287 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4962bf-131a-40c1-af60-21bf6f37513e None admin] PATH_INFO: `/v3/endpoints` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:18.771579 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4962bf-131a-40c1-af60-21bf6f37513e None admin] RBAC: Adding query filter params (interface=public, service_id=a8605039e90a41a38ed7b098d7b4a5de, region_id=RegionOne) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:00:18.771579 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4962bf-131a-40c1-af60-21bf6f37513e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:00:18.771579 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4962bf-131a-40c1-af60-21bf6f37513e None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:00:18.771579 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 534/1035] 10.222.0.55 () {56 vars in 1251 bytes} [Fri Sep 23 15:00:18 2022] GET /identity/v3/endpoints?service_id=a8605039e90a41a38ed7b098d7b4a5de&interface=public®ion_id=RegionOne => generated 194 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 15:00:18.772865 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4962bf-131a-40c1-af60-21bf6f37513e None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:00:18.811262 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:00:18.860622 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757744f5-0aca-40fc-a3fe-d463dc9728a1 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:00:18.878607 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4962bf-131a-40c1-af60-21bf6f37513e None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:00:18.889394 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:00:18.898073 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737f3f7f-e127-4447-acf2-4a1a8f06af4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:00:18.898739 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737f3f7f-e127-4447-acf2-4a1a8f06af4a None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:18.898965 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737f3f7f-e127-4447-acf2-4a1a8f06af4a None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:18.899234 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737f3f7f-e127-4447-acf2-4a1a8f06af4a None admin] PATH_INFO: `/v3/domains` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:18.899910 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737f3f7f-e127-4447-acf2-4a1a8f06af4a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:00:18.900237 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737f3f7f-e127-4447-acf2-4a1a8f06af4a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:00:18.901607 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737f3f7f-e127-4447-acf2-4a1a8f06af4a None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:00:18.916387 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 502/1036] 10.222.0.55 () {56 vars in 1109 bytes} [Fri Sep 23 15:00:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 15:00:18.918011 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab302477-5f45-411b-af4a-e14fb733a809 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:00:18.918711 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab302477-5f45-411b-af4a-e14fb733a809 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:18.919301 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab302477-5f45-411b-af4a-e14fb733a809 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:18.919434 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab302477-5f45-411b-af4a-e14fb733a809 None admin] PATH_INFO: `/v3/services` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:18.920075 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab302477-5f45-411b-af4a-e14fb733a809 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:00:18.920249 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737f3f7f-e127-4447-acf2-4a1a8f06af4a None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:00:18.921555 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab302477-5f45-411b-af4a-e14fb733a809 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:00:18.929204 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 535/1037] 10.222.0.55 () {56 vars in 1087 bytes} [Fri Sep 23 15:00:18 2022] GET /identity/v3/services => generated 1957 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:00:19.033521 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737f3f7f-e127-4447-acf2-4a1a8f06af4a None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:00:19.052387 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:00:19.090413 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa967e3b-d184-4096-a1a4-3f026ba6e30f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:00:19.091079 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa967e3b-d184-4096-a1a4-3f026ba6e30f None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:19.091354 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa967e3b-d184-4096-a1a4-3f026ba6e30f None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:19.091629 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa967e3b-d184-4096-a1a4-3f026ba6e30f None admin] PATH_INFO: `/v3/users/nova` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:19.100044 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa967e3b-d184-4096-a1a4-3f026ba6e30f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:00:19.101511 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa967e3b-d184-4096-a1a4-3f026ba6e30f None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:00:19.117013 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application [None req-fa967e3b-d184-4096-a1a4-3f026ba6e30f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 23 15:00:19.117013 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:00:19.117013 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:00:19.117013 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:00:19.117013 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:00:19.117013 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:00:19.117013 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:00:19.117013 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:00:19.117013 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:00:19.117013 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:00:19.117013 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:00:19.117013 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:00:19.117013 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 23 15:00:19.117013 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 23 15:00:19.117013 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 23 15:00:19.117013 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 23 15:00:19.118084 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:00:19.118084 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:00:19.118084 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 23 15:00:19.118084 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 15:00:19.118084 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 23 15:00:19.118084 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 15:00:19.118084 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:00:19.118084 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:00:19.118084 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:00:19.118084 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:00:19.118084 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:00:19.118084 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application with Lock( Sep 23 15:00:19.118084 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:00:19.118084 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:00:19.118084 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:00:19.118084 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:00:19.118084 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:00:19.118084 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:00:19.120451 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:00:19.120451 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:00:19.120451 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 23 15:00:19.120451 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 23 15:00:19.120451 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 23 15:00:19.120451 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 23 15:00:19.120451 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 23 15:00:19.120451 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 23 15:00:19.120451 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 23 15:00:19.120451 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Sep 23 15:00:19.120451 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 503/1038] 10.222.0.55 () {56 vars in 1093 bytes} [Fri Sep 23 15:00:18 2022] GET /identity/v3/users/nova => generated 82 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 23 15:00:19.125146 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab302477-5f45-411b-af4a-e14fb733a809 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:00:19.236221 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab302477-5f45-411b-af4a-e14fb733a809 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:00:19.271790 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070becf8-4622-4f86-a0b4-af6f6085c818 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:00:19.272685 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070becf8-4622-4f86-a0b4-af6f6085c818 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:19.273127 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070becf8-4622-4f86-a0b4-af6f6085c818 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:19.273891 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070becf8-4622-4f86-a0b4-af6f6085c818 None admin] PATH_INFO: `/v3/users` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:19.275431 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070becf8-4622-4f86-a0b4-af6f6085c818 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:00:19.275936 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070becf8-4622-4f86-a0b4-af6f6085c818 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:00:19.277565 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070becf8-4622-4f86-a0b4-af6f6085c818 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:00:19.296972 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 536/1039] 10.222.0.55 () {56 vars in 1133 bytes} [Fri Sep 23 15:00:19 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:00:19.302330 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa967e3b-d184-4096-a1a4-3f026ba6e30f None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:00:19.411115 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa967e3b-d184-4096-a1a4-3f026ba6e30f None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:00:19.447121 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99784b6-0bd3-4b8b-9aed-711d657f7cc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:00:19.447661 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99784b6-0bd3-4b8b-9aed-711d657f7cc9 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:19.447918 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99784b6-0bd3-4b8b-9aed-711d657f7cc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:19.448223 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99784b6-0bd3-4b8b-9aed-711d657f7cc9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:19.492377 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:00:19.517638 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99784b6-0bd3-4b8b-9aed-711d657f7cc9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:00:19.519158 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99784b6-0bd3-4b8b-9aed-711d657f7cc9 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:00:19.621370 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 504/1040] 10.222.0.55 () {58 vars in 1282 bytes} [Fri Sep 23 15:00:19 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 23 15:00:19.627093 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070becf8-4622-4f86-a0b4-af6f6085c818 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:00:19.661145 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:00:19.735859 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070becf8-4622-4f86-a0b4-af6f6085c818 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:00:19.774569 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23db2f1-01aa-4c94-81ac-96653aa93964 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:00:19.775737 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23db2f1-01aa-4c94-81ac-96653aa93964 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:19.776034 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23db2f1-01aa-4c94-81ac-96653aa93964 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:19.776309 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23db2f1-01aa-4c94-81ac-96653aa93964 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:19.786863 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23db2f1-01aa-4c94-81ac-96653aa93964 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:00:19.788313 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23db2f1-01aa-4c94-81ac-96653aa93964 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:00:19.803777 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-f23db2f1-01aa-4c94-81ac-96653aa93964 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 15:00:19.803777 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:00:19.803777 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:00:19.803777 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:00:19.803777 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:00:19.803777 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:00:19.803777 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:00:19.803777 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:00:19.803777 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:00:19.803777 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:00:19.803777 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:00:19.803777 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:00:19.803777 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 15:00:19.803777 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 15:00:19.803777 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 15:00:19.803777 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 15:00:19.804991 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:00:19.804991 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:00:19.804991 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:00:19.804991 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:00:19.804991 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:00:19.804991 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:00:19.804991 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:00:19.804991 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Sep 23 15:00:19.804991 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:00:19.804991 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:00:19.804991 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:00:19.804991 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:00:19.804991 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:00:19.804991 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:00:19.804991 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:00:19.804991 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:00:19.804991 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 15:00:19.806600 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 15:00:19.806600 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 15:00:19.806600 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Sep 23 15:00:19.806600 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 537/1041] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Sep 23 15:00:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:00:19.812846 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99784b6-0bd3-4b8b-9aed-711d657f7cc9 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:00:19.893101 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:00:19.926131 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99784b6-0bd3-4b8b-9aed-711d657f7cc9 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:00:19.963731 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74474293-21e7-4352-9753-ed18b2e33969 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:00:19.964373 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74474293-21e7-4352-9753-ed18b2e33969 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:19.964655 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74474293-21e7-4352-9753-ed18b2e33969 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:19.964929 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74474293-21e7-4352-9753-ed18b2e33969 None admin] PATH_INFO: `/v3/domains` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:19.965582 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74474293-21e7-4352-9753-ed18b2e33969 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:00:19.965879 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74474293-21e7-4352-9753-ed18b2e33969 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:00:19.967231 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74474293-21e7-4352-9753-ed18b2e33969 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:00:19.979485 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 505/1042] 10.222.0.55 () {56 vars in 1109 bytes} [Fri Sep 23 15:00:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:00:19.984913 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23db2f1-01aa-4c94-81ac-96653aa93964 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:00:20.026190 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7036f72-ce7b-4ffb-b0af-58bfe0fba7bc None None] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:20.026385 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7036f72-ce7b-4ffb-b0af-58bfe0fba7bc None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:20.026693 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7036f72-ce7b-4ffb-b0af-58bfe0fba7bc None None] PATH_INFO: `/` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:20.027952 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 506/1043] 10.222.0.55 () {54 vars in 927 bytes} [Fri Sep 23 15:00:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 23 15:00:20.033244 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28aa0493-b60f-4926-90af-af4f9377ba84 None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:20.033533 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28aa0493-b60f-4926-90af-af4f9377ba84 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:20.033807 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28aa0493-b60f-4926-90af-af4f9377ba84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:20.052814 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-28aa0493-b60f-4926-90af-af4f9377ba84 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:00:20.054038 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-28aa0493-b60f-4926-90af-af4f9377ba84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:00:20.085605 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:00:20.106070 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23db2f1-01aa-4c94-81ac-96653aa93964 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:00:20.133216 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 507/1044] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Sep 23 15:00:20 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 23 15:00:20.144319 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31919ea-f09f-480e-ac14-6efce7d90ee2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:00:20.145033 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31919ea-f09f-480e-ac14-6efce7d90ee2 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:20.145380 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31919ea-f09f-480e-ac14-6efce7d90ee2 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:20.145718 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31919ea-f09f-480e-ac14-6efce7d90ee2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:20.152876 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31919ea-f09f-480e-ac14-6efce7d90ee2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:00:20.154436 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31919ea-f09f-480e-ac14-6efce7d90ee2 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:00:20.168738 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-d31919ea-f09f-480e-ac14-6efce7d90ee2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 15:00:20.168738 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:00:20.168738 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:00:20.168738 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:00:20.168738 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:00:20.168738 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:00:20.168738 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:00:20.168738 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:00:20.168738 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:00:20.168738 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:00:20.168738 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:00:20.168738 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:00:20.168738 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 23 15:00:20.168738 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 23 15:00:20.168738 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 23 15:00:20.169967 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 23 15:00:20.169967 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:00:20.169967 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:00:20.169967 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:00:20.169967 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:00:20.169967 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:00:20.169967 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:00:20.169967 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:00:20.169967 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Sep 23 15:00:20.169967 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:00:20.169967 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:00:20.169967 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:00:20.169967 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:00:20.169967 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:00:20.169967 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:00:20.169967 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:00:20.169967 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:00:20.171120 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 23 15:00:20.171120 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 23 15:00:20.171120 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 23 15:00:20.171120 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 23 15:00:20.171120 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 23 15:00:20.171120 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 23 15:00:20.171120 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 15:00:20.171120 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Sep 23 15:00:20.171898 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 538/1045] 10.222.0.55 () {56 vars in 1111 bytes} [Fri Sep 23 15:00:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:00:20.177654 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28aa0493-b60f-4926-90af-af4f9377ba84 None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:00:20.251870 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:00:20.290946 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28aa0493-b60f-4926-90af-af4f9377ba84 None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:00:20.325922 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5f71a8-79bb-492f-81d6-2741b1d7c0c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:00:20.326641 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5f71a8-79bb-492f-81d6-2741b1d7c0c7 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:20.326990 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5f71a8-79bb-492f-81d6-2741b1d7c0c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:20.327329 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5f71a8-79bb-492f-81d6-2741b1d7c0c7 None admin] PATH_INFO: `/v3/projects` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:20.328117 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5f71a8-79bb-492f-81d6-2741b1d7c0c7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:00:20.328535 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5f71a8-79bb-492f-81d6-2741b1d7c0c7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:00:20.330022 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5f71a8-79bb-492f-81d6-2741b1d7c0c7 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:00:20.346372 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 508/1046] 10.222.0.55 () {56 vars in 1148 bytes} [Fri Sep 23 15:00:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:00:20.348175 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31919ea-f09f-480e-ac14-6efce7d90ee2 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:00:20.410093 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5f71a8-79bb-492f-81d6-2741b1d7c0c7 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:00:20.451027 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31919ea-f09f-480e-ac14-6efce7d90ee2 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:00:20.485039 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3dbf7b3-1385-4078-86dd-029a9777eda1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:00:20.485648 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dbf7b3-1385-4078-86dd-029a9777eda1 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:20.485926 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dbf7b3-1385-4078-86dd-029a9777eda1 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:20.486197 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dbf7b3-1385-4078-86dd-029a9777eda1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:20.486919 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3dbf7b3-1385-4078-86dd-029a9777eda1 None admin] RBAC: Adding query filter params (role.id=f2f7d48ece4f42f682922c1f1beefbd9, scope.project.id=87b3e9d4b6474bd4a04ba59ef52bf3e2, user.id=b3c6c7a1325a47be924268e1aa1a7a18) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:00:20.487226 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3dbf7b3-1385-4078-86dd-029a9777eda1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:00:20.488713 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3dbf7b3-1385-4078-86dd-029a9777eda1 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:00:20.504014 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 539/1047] 10.222.0.55 () {56 vars in 1374 bytes} [Fri Sep 23 15:00:20 2022] GET /identity/v3/role_assignments?user.id=b3c6c7a1325a47be924268e1aa1a7a18&scope.project.id=87b3e9d4b6474bd4a04ba59ef52bf3e2&role.id=f2f7d48ece4f42f682922c1f1beefbd9 => generated 259 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 15:00:20.524029 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5f71a8-79bb-492f-81d6-2741b1d7c0c7 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:00:20.554612 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:00:20.566862 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2362563f-51a5-41dd-9cd0-8b3ccb12e33d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:00:20.569029 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2362563f-51a5-41dd-9cd0-8b3ccb12e33d None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:20.569029 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2362563f-51a5-41dd-9cd0-8b3ccb12e33d None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:20.569029 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2362563f-51a5-41dd-9cd0-8b3ccb12e33d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:20.581493 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2362563f-51a5-41dd-9cd0-8b3ccb12e33d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 15:00:20.586645 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application [None req-2362563f-51a5-41dd-9cd0-8b3ccb12e33d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 23 15:00:20.586645 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:00:20.586645 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:00:20.586645 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:00:20.586645 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:00:20.586645 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:00:20.586645 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:00:20.586645 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:00:20.586645 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:00:20.586645 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:00:20.586645 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:00:20.586645 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:00:20.586645 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 15:00:20.586645 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 15:00:20.586645 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 15:00:20.586645 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 15:00:20.588847 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 15:00:20.588847 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 15:00:20.588847 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 15:00:20.588847 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 15:00:20.588847 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:00:20.588847 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:00:20.588847 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:00:20.588847 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:00:20.588847 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:00:20.588847 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:00:20.588847 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:00:20.588847 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application with Lock( Sep 23 15:00:20.588847 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:00:20.588847 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:00:20.588847 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:00:20.588847 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:00:20.589990 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:00:20.589990 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:00:20.589990 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:00:20.589990 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:00:20.589990 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 15:00:20.589990 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 15:00:20.589990 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 15:00:20.589990 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 15:00:20.589990 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 15:00:20.589990 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 15:00:20.589990 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 23 15:00:20.589990 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Sep 23 15:00:20.590961 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 509/1048] 10.222.0.55 () {56 vars in 1096 bytes} [Fri Sep 23 15:00:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 23 15:00:20.596722 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3dbf7b3-1385-4078-86dd-029a9777eda1 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:00:20.607582 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:00:20.715804 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3dbf7b3-1385-4078-86dd-029a9777eda1 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:00:20.754705 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e61bd3-b9f4-44b3-9fb4-e14d86edd697 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:00:20.755471 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e61bd3-b9f4-44b3-9fb4-e14d86edd697 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:20.755855 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e61bd3-b9f4-44b3-9fb4-e14d86edd697 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:20.756276 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e61bd3-b9f4-44b3-9fb4-e14d86edd697 None admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:20.757085 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e61bd3-b9f4-44b3-9fb4-e14d86edd697 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:00:20.757486 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e61bd3-b9f4-44b3-9fb4-e14d86edd697 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:00:20.759800 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e61bd3-b9f4-44b3-9fb4-e14d86edd697 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:00:20.774501 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 540/1049] 10.222.0.55 () {56 vars in 1099 bytes} [Fri Sep 23 15:00:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:00:20.776038 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2362563f-51a5-41dd-9cd0-8b3ccb12e33d None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:00:20.885734 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2362563f-51a5-41dd-9cd0-8b3ccb12e33d None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:00:20.920706 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6ce07f-bebd-4ccd-9d35-98802c4c66b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:00:20.921347 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6ce07f-bebd-4ccd-9d35-98802c4c66b6 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:20.921629 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6ce07f-bebd-4ccd-9d35-98802c4c66b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:20.921902 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6ce07f-bebd-4ccd-9d35-98802c4c66b6 None admin] PATH_INFO: `/v3/users/7b2d264e3df84671a429209719cd6b9b` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:20.937093 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6ce07f-bebd-4ccd-9d35-98802c4c66b6 None admin] RBAC: Authorizing `identity:get_user(user_id=7b2d264e3df84671a429209719cd6b9b)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:00:20.938545 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6ce07f-bebd-4ccd-9d35-98802c4c66b6 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:00:20.944884 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 510/1050] 10.222.0.55 () {56 vars in 1177 bytes} [Fri Sep 23 15:00:20 2022] GET /identity/v3/users/7b2d264e3df84671a429209719cd6b9b => generated 285 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:00:20.946358 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e61bd3-b9f4-44b3-9fb4-e14d86edd697 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:00:21.021338 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:00:21.058444 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e61bd3-b9f4-44b3-9fb4-e14d86edd697 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:00:21.100206 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f175f9a-083e-4de5-a69e-3b5af3b0f5ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:00:21.100974 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f175f9a-083e-4de5-a69e-3b5af3b0f5ae None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:21.101358 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f175f9a-083e-4de5-a69e-3b5af3b0f5ae None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:21.101729 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f175f9a-083e-4de5-a69e-3b5af3b0f5ae None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:21.135950 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f490d52-e316-4c6b-ae25-aa4a9f7aab51 None None] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:21.136500 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f490d52-e316-4c6b-ae25-aa4a9f7aab51 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:21.136500 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f490d52-e316-4c6b-ae25-aa4a9f7aab51 None None] PATH_INFO: `/` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:21.138012 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 511/1051] 10.222.0.55 () {54 vars in 927 bytes} [Fri Sep 23 15:00:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 23 15:00:21.142933 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ac6aa0-4f13-4c58-a761-082e7e50262b None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:21.143211 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ac6aa0-4f13-4c58-a761-082e7e50262b None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:21.143480 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ac6aa0-4f13-4c58-a761-082e7e50262b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:21.162404 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-f0ac6aa0-4f13-4c58-a761-082e7e50262b None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:00:21.163792 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-f0ac6aa0-4f13-4c58-a761-082e7e50262b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:00:21.174023 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f175f9a-083e-4de5-a69e-3b5af3b0f5ae None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:00:21.176044 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f175f9a-083e-4de5-a69e-3b5af3b0f5ae None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:00:21.204642 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:00:21.237587 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 512/1052] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Sep 23 15:00:21 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 23 15:00:21.254169 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:00:21.295061 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 541/1053] 10.222.0.55 () {58 vars in 1282 bytes} [Fri Sep 23 15:00:20 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 23 15:00:21.296374 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ac6aa0-4f13-4c58-a761-082e7e50262b None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:00:21.405784 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ac6aa0-4f13-4c58-a761-082e7e50262b None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:00:21.443987 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4613dcd9-38e7-4b90-a5a9-471f111495cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:00:21.444826 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4613dcd9-38e7-4b90-a5a9-471f111495cf None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:21.445205 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4613dcd9-38e7-4b90-a5a9-471f111495cf None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:21.445593 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4613dcd9-38e7-4b90-a5a9-471f111495cf None admin] PATH_INFO: `/v3/projects/8df0b9101ae94fb697db28a50cd90aeb` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:21.447570 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4613dcd9-38e7-4b90-a5a9-471f111495cf None admin] RBAC: Authorizing `identity:get_project(project_id=8df0b9101ae94fb697db28a50cd90aeb)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:00:21.449218 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4613dcd9-38e7-4b90-a5a9-471f111495cf None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:00:21.456127 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 513/1054] 10.222.0.55 () {56 vars in 1186 bytes} [Fri Sep 23 15:00:21 2022] GET /identity/v3/projects/8df0b9101ae94fb697db28a50cd90aeb => generated 311 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 15:00:21.457273 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f175f9a-083e-4de5-a69e-3b5af3b0f5ae None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:00:21.515995 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4613dcd9-38e7-4b90-a5a9-471f111495cf None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:00:21.559399 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:00:21.567266 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f175f9a-083e-4de5-a69e-3b5af3b0f5ae None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:00:21.605221 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da1d265-2fc7-4818-8878-090d5304100a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:00:21.605835 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da1d265-2fc7-4818-8878-090d5304100a None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:21.606117 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da1d265-2fc7-4818-8878-090d5304100a None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:21.606390 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da1d265-2fc7-4818-8878-090d5304100a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:21.607111 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da1d265-2fc7-4818-8878-090d5304100a None admin] RBAC: Adding query filter params (role.id=8e2919e79df040e8926f21607818b58c, scope.project.id=8df0b9101ae94fb697db28a50cd90aeb, user.id=7b2d264e3df84671a429209719cd6b9b) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:00:21.607422 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da1d265-2fc7-4818-8878-090d5304100a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:00:21.608902 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da1d265-2fc7-4818-8878-090d5304100a None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:00:21.624008 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 542/1055] 10.222.0.55 () {56 vars in 1374 bytes} [Fri Sep 23 15:00:21 2022] GET /identity/v3/role_assignments?user.id=7b2d264e3df84671a429209719cd6b9b&scope.project.id=8df0b9101ae94fb697db28a50cd90aeb&role.id=8e2919e79df040e8926f21607818b58c => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 15:00:21.628231 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4613dcd9-38e7-4b90-a5a9-471f111495cf None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:00:21.668923 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32574ee-44a4-41e2-b9ee-4d25a4109851 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:00:21.669495 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32574ee-44a4-41e2-b9ee-4d25a4109851 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:21.669783 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32574ee-44a4-41e2-b9ee-4d25a4109851 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:21.670058 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32574ee-44a4-41e2-b9ee-4d25a4109851 None admin] PATH_INFO: `/v3/services/image` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:21.670658 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32574ee-44a4-41e2-b9ee-4d25a4109851 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:00:21.672131 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32574ee-44a4-41e2-b9ee-4d25a4109851 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:00:21.682384 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application [None req-f32574ee-44a4-41e2-b9ee-4d25a4109851 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 23 15:00:21.682384 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:00:21.682384 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 23 15:00:21.682384 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 23 15:00:21.682384 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 23 15:00:21.682384 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 23 15:00:21.682384 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 23 15:00:21.682384 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 23 15:00:21.682384 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 23 15:00:21.682384 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Sep 23 15:00:21.682384 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 23 15:00:21.682384 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Sep 23 15:00:21.682384 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:00:21.682384 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:00:21.682384 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:00:21.682384 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:00:21.682384 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:00:21.683513 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:00:21.683513 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:00:21.683513 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:00:21.683513 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:00:21.683513 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:00:21.683513 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:00:21.683513 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 23 15:00:21.683513 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 23 15:00:21.683513 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 23 15:00:21.683513 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 23 15:00:21.683513 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:00:21.683513 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:00:21.683513 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:00:21.683513 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:00:21.683513 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:00:21.683513 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:00:21.684610 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:00:21.684610 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application with Lock( Sep 23 15:00:21.684610 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:00:21.684610 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:00:21.684610 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:00:21.684610 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:00:21.684610 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:00:21.684610 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:00:21.684610 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:00:21.684610 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:00:21.684610 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 23 15:00:21.684610 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 23 15:00:21.684610 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 23 15:00:21.684610 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Sep 23 15:00:21.688339 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 514/1056] 10.222.0.55 () {56 vars in 1105 bytes} [Fri Sep 23 15:00:21 2022] GET /identity/v3/services/image => generated 86 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 23 15:00:21.689810 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da1d265-2fc7-4818-8878-090d5304100a None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:00:21.726414 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:00:21.807397 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da1d265-2fc7-4818-8878-090d5304100a None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:00:21.841780 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe20367-a1c4-40b9-9fc5-4decedd8a474 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:00:21.842394 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe20367-a1c4-40b9-9fc5-4decedd8a474 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:21.842675 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe20367-a1c4-40b9-9fc5-4decedd8a474 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:21.842945 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe20367-a1c4-40b9-9fc5-4decedd8a474 None admin] PATH_INFO: `/v3/services` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:21.843573 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe20367-a1c4-40b9-9fc5-4decedd8a474 None admin] RBAC: Adding query filter params (name=image) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:00:21.843877 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe20367-a1c4-40b9-9fc5-4decedd8a474 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:00:21.845268 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe20367-a1c4-40b9-9fc5-4decedd8a474 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:00:21.850286 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 543/1057] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Sep 23 15:00:21 2022] GET /identity/v3/services?name=image => generated 122 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:00:21.856026 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32574ee-44a4-41e2-b9ee-4d25a4109851 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:00:21.969770 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32574ee-44a4-41e2-b9ee-4d25a4109851 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:00:21.972076 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:00:22.004949 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b791b0-9547-4cd3-945d-32d98fd22cc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:00:22.005558 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b791b0-9547-4cd3-945d-32d98fd22cc8 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:22.005837 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b791b0-9547-4cd3-945d-32d98fd22cc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:22.006109 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b791b0-9547-4cd3-945d-32d98fd22cc8 None admin] PATH_INFO: `/v3/services` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:22.006731 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b791b0-9547-4cd3-945d-32d98fd22cc8 None admin] RBAC: Adding query filter params (type=image) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:00:22.007038 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b791b0-9547-4cd3-945d-32d98fd22cc8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:00:22.008429 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b791b0-9547-4cd3-945d-32d98fd22cc8 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:00:22.018074 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 515/1058] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Sep 23 15:00:21 2022] GET /identity/v3/services?type=image => generated 350 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:00:22.019216 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe20367-a1c4-40b9-9fc5-4decedd8a474 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:00:22.170872 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:00:22.170872 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe20367-a1c4-40b9-9fc5-4decedd8a474 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:00:22.182710 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d738c58a-a877-4769-8e33-e33c29063c69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:00:22.183361 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d738c58a-a877-4769-8e33-e33c29063c69 None admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:22.183605 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d738c58a-a877-4769-8e33-e33c29063c69 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:22.183882 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d738c58a-a877-4769-8e33-e33c29063c69 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:22.184585 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d738c58a-a877-4769-8e33-e33c29063c69 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:00:22.185959 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d738c58a-a877-4769-8e33-e33c29063c69 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:00:22.333817 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 544/1059] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Sep 23 15:00:22 2022] POST /identity/v3/endpoints => generated 327 bytes in 315 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:00:22.734790 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea3e1ae-6c7a-4840-84d8-d1a1e4c44978 None None] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:22.735302 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea3e1ae-6c7a-4840-84d8-d1a1e4c44978 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:22.735302 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea3e1ae-6c7a-4840-84d8-d1a1e4c44978 None None] PATH_INFO: `/` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:22.736909 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 516/1060] 10.222.0.55 () {54 vars in 927 bytes} [Fri Sep 23 15:00:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:00:22.741988 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8347644-61e1-4600-b3df-0041b32fc5fc None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:22.742277 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8347644-61e1-4600-b3df-0041b32fc5fc None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:22.742548 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8347644-61e1-4600-b3df-0041b32fc5fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:22.762682 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-d8347644-61e1-4600-b3df-0041b32fc5fc None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:00:22.763906 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-d8347644-61e1-4600-b3df-0041b32fc5fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:00:22.842241 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 545/1061] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Sep 23 15:00:22 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 23 15:00:23.109271 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea3e1ae-6c7a-4840-84d8-d1a1e4c44978 None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:00:23.214163 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea3e1ae-6c7a-4840-84d8-d1a1e4c44978 None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:00:23.249837 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac37aba-2625-45af-a8f0-52aecc362236 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:00:23.250500 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac37aba-2625-45af-a8f0-52aecc362236 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:23.250775 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac37aba-2625-45af-a8f0-52aecc362236 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:23.251039 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac37aba-2625-45af-a8f0-52aecc362236 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:23.266218 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac37aba-2625-45af-a8f0-52aecc362236 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 15:00:23.270994 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application [None req-5ac37aba-2625-45af-a8f0-52aecc362236 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 23 15:00:23.270994 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:00:23.270994 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:00:23.270994 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:00:23.270994 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:00:23.270994 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:00:23.270994 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:00:23.270994 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:00:23.270994 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:00:23.270994 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:00:23.270994 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:00:23.270994 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:00:23.270994 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 15:00:23.270994 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 15:00:23.270994 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 15:00:23.270994 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 15:00:23.272002 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 15:00:23.272002 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 15:00:23.272002 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 15:00:23.272002 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 15:00:23.272002 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:00:23.272002 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:00:23.272002 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:00:23.272002 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:00:23.272002 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:00:23.272002 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:00:23.272002 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:00:23.272002 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application with Lock( Sep 23 15:00:23.272002 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:00:23.272002 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:00:23.272002 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:00:23.272002 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:00:23.272895 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:00:23.272895 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:00:23.272895 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:00:23.272895 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:00:23.272895 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 15:00:23.272895 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 15:00:23.272895 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 15:00:23.272895 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 15:00:23.272895 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 15:00:23.272895 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 15:00:23.272895 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 23 15:00:23.272895 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Sep 23 15:00:23.276088 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 517/1062] 10.222.0.55 () {56 vars in 1120 bytes} [Fri Sep 23 15:00:23 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:00:23.278369 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8347644-61e1-4600-b3df-0041b32fc5fc None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:00:23.362114 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:00:23.385583 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8347644-61e1-4600-b3df-0041b32fc5fc None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:00:23.420497 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b227e5-7bde-4d93-9b47-330fc3bdcc6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:00:23.421108 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b227e5-7bde-4d93-9b47-330fc3bdcc6c None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:23.421388 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b227e5-7bde-4d93-9b47-330fc3bdcc6c None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:23.421666 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b227e5-7bde-4d93-9b47-330fc3bdcc6c None admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:23.422368 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b227e5-7bde-4d93-9b47-330fc3bdcc6c None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:00:23.422672 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b227e5-7bde-4d93-9b47-330fc3bdcc6c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:00:23.424053 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b227e5-7bde-4d93-9b47-330fc3bdcc6c None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:00:23.438105 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 546/1063] 10.222.0.55 () {56 vars in 1115 bytes} [Fri Sep 23 15:00:23 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:00:23.439325 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac37aba-2625-45af-a8f0-52aecc362236 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:00:23.468766 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:00:23.542407 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac37aba-2625-45af-a8f0-52aecc362236 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:00:23.576353 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b42d32-cb65-4c88-8346-0c0cf4846fa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:00:23.576990 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b42d32-cb65-4c88-8346-0c0cf4846fa9 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:23.577270 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b42d32-cb65-4c88-8346-0c0cf4846fa9 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:23.577539 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b42d32-cb65-4c88-8346-0c0cf4846fa9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:23.588523 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b42d32-cb65-4c88-8346-0c0cf4846fa9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:00:23.589972 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b42d32-cb65-4c88-8346-0c0cf4846fa9 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:00:23.607070 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application [None req-34b42d32-cb65-4c88-8346-0c0cf4846fa9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 15:00:23.607070 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:00:23.607070 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:00:23.607070 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:00:23.607070 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:00:23.607070 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:00:23.607070 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:00:23.607070 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:00:23.607070 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:00:23.607070 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:00:23.607070 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:00:23.607070 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:00:23.607070 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 15:00:23.607070 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 15:00:23.607070 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 15:00:23.607070 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 15:00:23.613043 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:00:23.613043 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:00:23.613043 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:00:23.613043 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:00:23.613043 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:00:23.613043 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:00:23.613043 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:00:23.613043 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application with Lock( Sep 23 15:00:23.613043 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:00:23.613043 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:00:23.613043 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:00:23.613043 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:00:23.613043 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:00:23.613043 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:00:23.613043 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:00:23.613043 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:00:23.613043 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 15:00:23.614033 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 15:00:23.614033 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 15:00:23.614033 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Sep 23 15:00:23.614033 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 518/1064] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Sep 23 15:00:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 23 15:00:23.614289 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b227e5-7bde-4d93-9b47-330fc3bdcc6c None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:00:23.722433 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b227e5-7bde-4d93-9b47-330fc3bdcc6c None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:00:23.758646 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ca976f-c34d-4b5f-a983-2ba5f2c89e05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:00:23.759900 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ca976f-c34d-4b5f-a983-2ba5f2c89e05 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:23.760410 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ca976f-c34d-4b5f-a983-2ba5f2c89e05 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:23.760547 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ca976f-c34d-4b5f-a983-2ba5f2c89e05 None admin] PATH_INFO: `/v3/domains` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:23.761103 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ca976f-c34d-4b5f-a983-2ba5f2c89e05 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:00:23.761398 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ca976f-c34d-4b5f-a983-2ba5f2c89e05 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:00:23.762753 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ca976f-c34d-4b5f-a983-2ba5f2c89e05 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:00:23.771865 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:00:23.776015 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 547/1065] 10.222.0.55 () {56 vars in 1109 bytes} [Fri Sep 23 15:00:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 15:00:23.781568 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b42d32-cb65-4c88-8346-0c0cf4846fa9 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:00:23.806950 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8383eedb-1f88-4e03-9b35-a6024304fb84 None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:23.807272 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8383eedb-1f88-4e03-9b35-a6024304fb84 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:23.807504 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8383eedb-1f88-4e03-9b35-a6024304fb84 None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:23.809013 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 548/1066] 10.222.0.55 () {54 vars in 927 bytes} [Fri Sep 23 15:00:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:00:23.814597 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d4f95b-ebfe-42c8-b2cb-31148f195455 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:23.814962 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d4f95b-ebfe-42c8-b2cb-31148f195455 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:23.815202 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d4f95b-ebfe-42c8-b2cb-31148f195455 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:23.835056 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-d7d4f95b-ebfe-42c8-b2cb-31148f195455 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:00:23.836279 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-d7d4f95b-ebfe-42c8-b2cb-31148f195455 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:00:23.869881 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:00:23.894563 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b42d32-cb65-4c88-8346-0c0cf4846fa9 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:00:23.909470 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 549/1067] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Sep 23 15:00:23 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 23 15:00:23.939797 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399235a2-2eb7-4a6d-bec9-c1cb8a99523f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:00:23.949895 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399235a2-2eb7-4a6d-bec9-c1cb8a99523f None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:23.949895 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399235a2-2eb7-4a6d-bec9-c1cb8a99523f None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:23.949895 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399235a2-2eb7-4a6d-bec9-c1cb8a99523f None admin] PATH_INFO: `/v3/users/nova` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:23.949895 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399235a2-2eb7-4a6d-bec9-c1cb8a99523f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:00:23.950742 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399235a2-2eb7-4a6d-bec9-c1cb8a99523f None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:00:23.961893 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application [None req-399235a2-2eb7-4a6d-bec9-c1cb8a99523f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 23 15:00:23.961893 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:00:23.961893 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:00:23.961893 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:00:23.961893 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:00:23.961893 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:00:23.961893 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:00:23.961893 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:00:23.961893 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:00:23.961893 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:00:23.961893 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:00:23.961893 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:00:23.961893 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 23 15:00:23.961893 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 23 15:00:23.961893 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 23 15:00:23.961893 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 23 15:00:23.962892 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:00:23.962892 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:00:23.962892 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 23 15:00:23.962892 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 15:00:23.962892 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 23 15:00:23.962892 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 15:00:23.962892 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:00:23.962892 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:00:23.962892 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:00:23.962892 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:00:23.962892 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:00:23.962892 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application with Lock( Sep 23 15:00:23.962892 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:00:23.962892 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:00:23.962892 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:00:23.962892 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:00:23.962892 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:00:23.962892 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:00:23.963957 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:00:23.963957 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:00:23.963957 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 23 15:00:23.963957 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 23 15:00:23.963957 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 23 15:00:23.963957 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 23 15:00:23.963957 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 23 15:00:23.963957 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 23 15:00:23.963957 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 23 15:00:23.963957 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Sep 23 15:00:23.963957 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 519/1068] 10.222.0.55 () {56 vars in 1093 bytes} [Fri Sep 23 15:00:23 2022] GET /identity/v3/users/nova => generated 82 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:00:23.969176 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d4f95b-ebfe-42c8-b2cb-31148f195455 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:00:24.081708 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d4f95b-ebfe-42c8-b2cb-31148f195455 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:00:24.113527 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:00:24.120384 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430b9369-9625-4979-b6e6-790d25dfd1e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:00:24.120935 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430b9369-9625-4979-b6e6-790d25dfd1e7 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:24.121214 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430b9369-9625-4979-b6e6-790d25dfd1e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:24.121487 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430b9369-9625-4979-b6e6-790d25dfd1e7 None admin] PATH_INFO: `/v3/users` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:24.122210 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430b9369-9625-4979-b6e6-790d25dfd1e7 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:00:24.122513 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430b9369-9625-4979-b6e6-790d25dfd1e7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:00:24.123948 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430b9369-9625-4979-b6e6-790d25dfd1e7 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:00:24.146662 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 550/1069] 10.222.0.55 () {56 vars in 1133 bytes} [Fri Sep 23 15:00:23 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:00:24.147806 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399235a2-2eb7-4a6d-bec9-c1cb8a99523f None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:00:24.192907 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430b9369-9625-4979-b6e6-790d25dfd1e7 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:00:24.264844 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399235a2-2eb7-4a6d-bec9-c1cb8a99523f None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:00:24.301266 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430b9369-9625-4979-b6e6-790d25dfd1e7 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:00:24.301816 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990e4e54-0816-43d8-9ac0-3ce4e0bc97b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:00:24.302389 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990e4e54-0816-43d8-9ac0-3ce4e0bc97b4 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:24.302670 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990e4e54-0816-43d8-9ac0-3ce4e0bc97b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:24.302945 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990e4e54-0816-43d8-9ac0-3ce4e0bc97b4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:24.312835 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:00:24.314865 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990e4e54-0816-43d8-9ac0-3ce4e0bc97b4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:00:24.316681 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990e4e54-0816-43d8-9ac0-3ce4e0bc97b4 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:00:24.332540 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application [None req-990e4e54-0816-43d8-9ac0-3ce4e0bc97b4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 15:00:24.332540 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:00:24.332540 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:00:24.332540 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:00:24.332540 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:00:24.332540 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:00:24.332540 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:00:24.332540 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:00:24.332540 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:00:24.332540 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:00:24.332540 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:00:24.332540 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:00:24.332540 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 15:00:24.332540 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 15:00:24.332540 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 15:00:24.332540 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 15:00:24.333620 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:00:24.333620 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:00:24.333620 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:00:24.333620 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:00:24.333620 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:00:24.333620 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:00:24.333620 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:00:24.333620 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application with Lock( Sep 23 15:00:24.333620 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:00:24.333620 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:00:24.333620 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:00:24.333620 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:00:24.333620 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:00:24.333620 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:00:24.333620 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:00:24.333620 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:00:24.333620 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 15:00:24.334592 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 15:00:24.334592 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 15:00:24.334592 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Sep 23 15:00:24.334592 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 520/1070] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Sep 23 15:00:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:00:24.339877 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990e4e54-0816-43d8-9ac0-3ce4e0bc97b4 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:00:24.348148 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7a27c6-2168-4c6e-ab29-2ddff79a7c2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:00:24.348797 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7a27c6-2168-4c6e-ab29-2ddff79a7c2a None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:24.349081 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7a27c6-2168-4c6e-ab29-2ddff79a7c2a None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:24.349355 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7a27c6-2168-4c6e-ab29-2ddff79a7c2a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:24.361128 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7a27c6-2168-4c6e-ab29-2ddff79a7c2a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 15:00:24.365950 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-ac7a27c6-2168-4c6e-ab29-2ddff79a7c2a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 23 15:00:24.365950 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:00:24.365950 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:00:24.365950 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:00:24.365950 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:00:24.365950 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:00:24.365950 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:00:24.365950 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:00:24.365950 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:00:24.365950 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:00:24.365950 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:00:24.365950 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:00:24.365950 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 15:00:24.365950 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 15:00:24.365950 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 15:00:24.365950 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 15:00:24.366925 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 15:00:24.366925 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 15:00:24.366925 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 15:00:24.366925 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 15:00:24.366925 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:00:24.366925 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:00:24.366925 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:00:24.366925 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:00:24.366925 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:00:24.366925 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:00:24.366925 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:00:24.366925 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Sep 23 15:00:24.366925 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:00:24.366925 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:00:24.366925 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:00:24.366925 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:00:24.367786 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:00:24.367786 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:00:24.367786 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:00:24.367786 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:00:24.367786 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 15:00:24.367786 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 15:00:24.367786 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 15:00:24.367786 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 15:00:24.367786 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 15:00:24.367786 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 15:00:24.367786 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 23 15:00:24.367786 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Sep 23 15:00:24.367786 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 551/1071] 10.222.0.55 () {56 vars in 1096 bytes} [Fri Sep 23 15:00:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 23 15:00:24.373670 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7a27c6-2168-4c6e-ab29-2ddff79a7c2a None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:00:24.433883 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:00:24.451452 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990e4e54-0816-43d8-9ac0-3ce4e0bc97b4 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:00:24.494151 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722be2ce-025f-4819-8ac4-5366d26da41b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:00:24.494833 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722be2ce-025f-4819-8ac4-5366d26da41b None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:24.495273 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722be2ce-025f-4819-8ac4-5366d26da41b None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:24.495575 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722be2ce-025f-4819-8ac4-5366d26da41b None admin] PATH_INFO: `/v3/domains` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:24.496233 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722be2ce-025f-4819-8ac4-5366d26da41b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:00:24.496538 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722be2ce-025f-4819-8ac4-5366d26da41b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:00:24.497580 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7a27c6-2168-4c6e-ab29-2ddff79a7c2a None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:00:24.497910 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722be2ce-025f-4819-8ac4-5366d26da41b None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:00:24.515699 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 521/1072] 10.222.0.55 () {56 vars in 1109 bytes} [Fri Sep 23 15:00:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 15:00:24.516856 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722be2ce-025f-4819-8ac4-5366d26da41b None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:00:24.540834 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2089e496-0759-4ab4-ac59-00bac2a2a8a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:00:24.541570 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2089e496-0759-4ab4-ac59-00bac2a2a8a9 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:24.541852 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2089e496-0759-4ab4-ac59-00bac2a2a8a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:24.542127 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2089e496-0759-4ab4-ac59-00bac2a2a8a9 None admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:24.542797 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2089e496-0759-4ab4-ac59-00bac2a2a8a9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:00:24.543101 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2089e496-0759-4ab4-ac59-00bac2a2a8a9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:00:24.544502 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2089e496-0759-4ab4-ac59-00bac2a2a8a9 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:00:24.558759 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 552/1073] 10.222.0.55 () {56 vars in 1099 bytes} [Fri Sep 23 15:00:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 15:00:24.559858 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2089e496-0759-4ab4-ac59-00bac2a2a8a9 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:00:24.579764 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:00:24.627879 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722be2ce-025f-4819-8ac4-5366d26da41b None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:00:24.663629 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fba2b9-594e-4b4c-90b6-875904f6ab2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:00:24.664296 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fba2b9-594e-4b4c-90b6-875904f6ab2f None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:24.664579 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fba2b9-594e-4b4c-90b6-875904f6ab2f None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:24.664851 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fba2b9-594e-4b4c-90b6-875904f6ab2f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:24.667658 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2089e496-0759-4ab4-ac59-00bac2a2a8a9 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:00:24.672348 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fba2b9-594e-4b4c-90b6-875904f6ab2f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:00:24.673809 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fba2b9-594e-4b4c-90b6-875904f6ab2f None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:00:24.678167 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:00:24.685270 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application [None req-73fba2b9-594e-4b4c-90b6-875904f6ab2f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 15:00:24.685270 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:00:24.685270 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:00:24.685270 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:00:24.685270 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:00:24.685270 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:00:24.685270 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:00:24.685270 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:00:24.685270 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:00:24.685270 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:00:24.685270 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:00:24.685270 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:00:24.685270 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 23 15:00:24.685270 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 23 15:00:24.685270 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 23 15:00:24.686205 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 23 15:00:24.686205 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:00:24.686205 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:00:24.686205 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:00:24.686205 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:00:24.686205 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:00:24.686205 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:00:24.686205 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:00:24.686205 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application with Lock( Sep 23 15:00:24.686205 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:00:24.686205 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:00:24.686205 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:00:24.686205 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:00:24.686205 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:00:24.686205 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:00:24.686205 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:00:24.686205 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:00:24.687154 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 23 15:00:24.687154 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 23 15:00:24.687154 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 23 15:00:24.687154 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 23 15:00:24.687154 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 23 15:00:24.687154 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 23 15:00:24.687154 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 15:00:24.687154 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Sep 23 15:00:24.687154 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 522/1074] 10.222.0.55 () {56 vars in 1111 bytes} [Fri Sep 23 15:00:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:00:24.689322 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c54952-1b59-41c5-a3de-c7fe8b475024 None None] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:24.689606 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c54952-1b59-41c5-a3de-c7fe8b475024 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:24.702007 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c54952-1b59-41c5-a3de-c7fe8b475024 None None] PATH_INFO: `/` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:24.702007 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 523/1075] 10.222.0.55 () {54 vars in 927 bytes} [Fri Sep 23 15:00:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:00:24.702007 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c54952-1b59-41c5-a3de-c7fe8b475024 None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:00:24.703378 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb8d69d-7737-4e30-bd14-33e1433827db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:00:24.704034 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb8d69d-7737-4e30-bd14-33e1433827db None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:24.704319 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb8d69d-7737-4e30-bd14-33e1433827db None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:24.704593 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb8d69d-7737-4e30-bd14-33e1433827db None admin] PATH_INFO: `/v3/users/7b2d264e3df84671a429209719cd6b9b` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:24.705699 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb8d69d-7737-4e30-bd14-33e1433827db None admin] RBAC: Authorizing `identity:get_user(user_id=7b2d264e3df84671a429209719cd6b9b)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:00:24.707223 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb8d69d-7737-4e30-bd14-33e1433827db None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:00:24.708820 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 553/1076] 10.222.0.55 () {56 vars in 1177 bytes} [Fri Sep 23 15:00:24 2022] GET /identity/v3/users/7b2d264e3df84671a429209719cd6b9b => generated 285 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 15:00:24.709982 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4cb1a4-833f-4181-b34c-88b0573ae926 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:24.710264 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4cb1a4-833f-4181-b34c-88b0573ae926 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:24.710633 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4cb1a4-833f-4181-b34c-88b0573ae926 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:24.729842 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-db4cb1a4-833f-4181-b34c-88b0573ae926 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:00:24.731856 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-db4cb1a4-833f-4181-b34c-88b0573ae926 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:00:24.799858 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c54952-1b59-41c5-a3de-c7fe8b475024 None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:00:24.800840 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 554/1077] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Sep 23 15:00:24 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 23 15:00:24.801687 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4cb1a4-833f-4181-b34c-88b0573ae926 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:00:24.814556 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:00:24.836690 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0776cf2f-bca4-4ad5-b567-13779c724185 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:00:24.838004 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0776cf2f-bca4-4ad5-b567-13779c724185 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:24.838331 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0776cf2f-bca4-4ad5-b567-13779c724185 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:24.838614 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0776cf2f-bca4-4ad5-b567-13779c724185 None admin] PATH_INFO: `/v3/projects` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:24.839351 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0776cf2f-bca4-4ad5-b567-13779c724185 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:00:24.839662 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0776cf2f-bca4-4ad5-b567-13779c724185 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:00:24.841220 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0776cf2f-bca4-4ad5-b567-13779c724185 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:00:24.853824 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 524/1078] 10.222.0.55 () {56 vars in 1148 bytes} [Fri Sep 23 15:00:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 15:00:24.859804 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0776cf2f-bca4-4ad5-b567-13779c724185 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:00:24.929216 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4cb1a4-833f-4181-b34c-88b0573ae926 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:00:24.940641 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:00:24.968957 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0776cf2f-bca4-4ad5-b567-13779c724185 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:00:24.970622 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5c8008-0189-41f3-88d3-6234a6b7306d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:00:24.971492 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5c8008-0189-41f3-88d3-6234a6b7306d None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:24.971779 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5c8008-0189-41f3-88d3-6234a6b7306d None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:24.972097 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5c8008-0189-41f3-88d3-6234a6b7306d None admin] PATH_INFO: `/v3/projects/8df0b9101ae94fb697db28a50cd90aeb` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:24.973235 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5c8008-0189-41f3-88d3-6234a6b7306d None admin] RBAC: Authorizing `identity:get_project(project_id=8df0b9101ae94fb697db28a50cd90aeb)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:00:24.974677 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5c8008-0189-41f3-88d3-6234a6b7306d None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:00:24.976432 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 555/1079] 10.222.0.55 () {56 vars in 1186 bytes} [Fri Sep 23 15:00:24 2022] GET /identity/v3/projects/8df0b9101ae94fb697db28a50cd90aeb => generated 311 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:00:24.982408 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5c8008-0189-41f3-88d3-6234a6b7306d None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:00:25.005376 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62c40d1-75b0-4b9d-8a7f-65ba8718f3d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:00:25.006045 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62c40d1-75b0-4b9d-8a7f-65ba8718f3d8 None admin] REQUEST_METHOD: `PUT` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:25.006318 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62c40d1-75b0-4b9d-8a7f-65ba8718f3d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:25.006588 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62c40d1-75b0-4b9d-8a7f-65ba8718f3d8 None admin] PATH_INFO: `/v3/projects/87b3e9d4b6474bd4a04ba59ef52bf3e2/users/b3c6c7a1325a47be924268e1aa1a7a18/roles/f2f7d48ece4f42f682922c1f1beefbd9` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:25.008747 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62c40d1-75b0-4b9d-8a7f-65ba8718f3d8 None admin] RBAC: Authorizing `identity:create_grant(project_id=87b3e9d4b6474bd4a04ba59ef52bf3e2, user_id=b3c6c7a1325a47be924268e1aa1a7a18, role_id=f2f7d48ece4f42f682922c1f1beefbd9)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:00:25.010727 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62c40d1-75b0-4b9d-8a7f-65ba8718f3d8 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:00:25.091858 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5c8008-0189-41f3-88d3-6234a6b7306d None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:00:25.108261 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 525/1080] 10.222.0.55 () {58 vars in 1439 bytes} [Fri Sep 23 15:00:24 2022] PUT /identity/v3/projects/87b3e9d4b6474bd4a04ba59ef52bf3e2/users/b3c6c7a1325a47be924268e1aa1a7a18/roles/f2f7d48ece4f42f682922c1f1beefbd9 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 15:00:25.109106 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62c40d1-75b0-4b9d-8a7f-65ba8718f3d8 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:00:25.128590 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca6fd7c-a0f1-43cb-a9ba-7c621cabc14d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:00:25.129285 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca6fd7c-a0f1-43cb-a9ba-7c621cabc14d None admin] REQUEST_METHOD: `PUT` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:25.129565 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca6fd7c-a0f1-43cb-a9ba-7c621cabc14d None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:25.129839 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca6fd7c-a0f1-43cb-a9ba-7c621cabc14d None admin] PATH_INFO: `/v3/projects/8df0b9101ae94fb697db28a50cd90aeb/users/7b2d264e3df84671a429209719cd6b9b/roles/8e2919e79df040e8926f21607818b58c` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:25.131871 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca6fd7c-a0f1-43cb-a9ba-7c621cabc14d None admin] RBAC: Authorizing `identity:create_grant(project_id=8df0b9101ae94fb697db28a50cd90aeb, user_id=7b2d264e3df84671a429209719cd6b9b, role_id=8e2919e79df040e8926f21607818b58c)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:00:25.133366 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca6fd7c-a0f1-43cb-a9ba-7c621cabc14d None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:00:25.156577 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:00:25.201599 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 556/1081] 10.222.0.55 () {58 vars in 1439 bytes} [Fri Sep 23 15:00:24 2022] PUT /identity/v3/projects/8df0b9101ae94fb697db28a50cd90aeb/users/7b2d264e3df84671a429209719cd6b9b/roles/8e2919e79df040e8926f21607818b58c => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 15:00:25.245548 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62c40d1-75b0-4b9d-8a7f-65ba8718f3d8 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:00:25.287517 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a719ba0-38fa-4e88-87cf-19c5a0de5c9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:00:25.288219 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a719ba0-38fa-4e88-87cf-19c5a0de5c9c None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:25.288501 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a719ba0-38fa-4e88-87cf-19c5a0de5c9c None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:25.288774 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a719ba0-38fa-4e88-87cf-19c5a0de5c9c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:25.374342 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a719ba0-38fa-4e88-87cf-19c5a0de5c9c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:00:25.375816 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a719ba0-38fa-4e88-87cf-19c5a0de5c9c None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:00:25.411308 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:00:25.493319 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 526/1082] 10.222.0.55 () {58 vars in 1282 bytes} [Fri Sep 23 15:00:25 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 23 15:00:25.500205 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca6fd7c-a0f1-43cb-a9ba-7c621cabc14d None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:00:25.537745 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:00:25.604703 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca6fd7c-a0f1-43cb-a9ba-7c621cabc14d None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:00:25.641351 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996663c3-6af9-4ed2-bcba-74a736987c71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:00:25.641986 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996663c3-6af9-4ed2-bcba-74a736987c71 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:25.642266 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996663c3-6af9-4ed2-bcba-74a736987c71 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:25.642537 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996663c3-6af9-4ed2-bcba-74a736987c71 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:25.653676 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996663c3-6af9-4ed2-bcba-74a736987c71 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:00:25.655435 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996663c3-6af9-4ed2-bcba-74a736987c71 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:00:25.680761 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-996663c3-6af9-4ed2-bcba-74a736987c71 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 15:00:25.680761 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:00:25.680761 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:00:25.680761 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:00:25.680761 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:00:25.680761 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:00:25.680761 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:00:25.680761 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:00:25.680761 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:00:25.680761 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:00:25.680761 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:00:25.680761 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:00:25.680761 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 15:00:25.680761 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 15:00:25.680761 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 15:00:25.680761 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 15:00:25.682504 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:00:25.682504 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:00:25.682504 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:00:25.682504 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:00:25.682504 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:00:25.682504 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:00:25.682504 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:00:25.682504 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Sep 23 15:00:25.682504 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:00:25.682504 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:00:25.682504 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:00:25.682504 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:00:25.682504 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:00:25.682504 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:00:25.682504 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:00:25.682504 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:00:25.682504 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 15:00:25.683414 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 15:00:25.683414 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 15:00:25.683414 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Sep 23 15:00:25.684216 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 557/1083] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Sep 23 15:00:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:00:25.688609 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a719ba0-38fa-4e88-87cf-19c5a0de5c9c None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:00:25.799183 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a719ba0-38fa-4e88-87cf-19c5a0de5c9c None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:00:25.830189 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:00:25.836033 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7ef52f-67fe-45f9-a2a8-2cdc9cc622d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:00:25.836652 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7ef52f-67fe-45f9-a2a8-2cdc9cc622d2 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:25.836935 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7ef52f-67fe-45f9-a2a8-2cdc9cc622d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:25.837241 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7ef52f-67fe-45f9-a2a8-2cdc9cc622d2 None admin] PATH_INFO: `/v3/domains` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:25.838469 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7ef52f-67fe-45f9-a2a8-2cdc9cc622d2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:00:25.838780 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7ef52f-67fe-45f9-a2a8-2cdc9cc622d2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:00:25.840171 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7ef52f-67fe-45f9-a2a8-2cdc9cc622d2 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:00:25.860011 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 527/1084] 10.222.0.55 () {56 vars in 1109 bytes} [Fri Sep 23 15:00:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:00:27.340350 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5792c7-1746-4806-90d7-8cf3d096c7a3 None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:27.340860 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5792c7-1746-4806-90d7-8cf3d096c7a3 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:27.340860 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5792c7-1746-4806-90d7-8cf3d096c7a3 None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:27.342200 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 558/1085] 10.222.0.55 () {54 vars in 927 bytes} [Fri Sep 23 15:00:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:00:27.347919 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8377eb3-cb87-47d3-b840-8c82c28512c8 None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:27.348359 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8377eb3-cb87-47d3-b840-8c82c28512c8 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:27.348633 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8377eb3-cb87-47d3-b840-8c82c28512c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:27.367720 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-b8377eb3-cb87-47d3-b840-8c82c28512c8 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:00:27.369762 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-b8377eb3-cb87-47d3-b840-8c82c28512c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:00:27.404112 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15544e92-ed2b-4bcc-b242-1edbb56a9221 None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:27.404401 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15544e92-ed2b-4bcc-b242-1edbb56a9221 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:27.404683 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15544e92-ed2b-4bcc-b242-1edbb56a9221 None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:27.406004 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 559/1086] 10.222.0.55 () {54 vars in 927 bytes} [Fri Sep 23 15:00:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 23 15:00:27.411286 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d742ead-06d1-40c2-84c6-7b328cc62bb5 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:27.411569 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d742ead-06d1-40c2-84c6-7b328cc62bb5 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:27.411844 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d742ead-06d1-40c2-84c6-7b328cc62bb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:27.432581 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-5d742ead-06d1-40c2-84c6-7b328cc62bb5 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:00:27.433786 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-5d742ead-06d1-40c2-84c6-7b328cc62bb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:00:27.440539 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 528/1087] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Sep 23 15:00:27 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 23 15:00:27.502913 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 560/1088] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Sep 23 15:00:27 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 23 15:00:27.708596 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8377eb3-cb87-47d3-b840-8c82c28512c8 None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:00:27.778090 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d742ead-06d1-40c2-84c6-7b328cc62bb5 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:00:27.798250 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:00:27.809476 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8377eb3-cb87-47d3-b840-8c82c28512c8 None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:00:27.832960 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:00:27.845202 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e22a7f-ce60-4d81-8b6b-0cb934a47cd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:00:27.845838 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e22a7f-ce60-4d81-8b6b-0cb934a47cd0 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:27.846118 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e22a7f-ce60-4d81-8b6b-0cb934a47cd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:27.846385 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e22a7f-ce60-4d81-8b6b-0cb934a47cd0 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:27.856592 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e22a7f-ce60-4d81-8b6b-0cb934a47cd0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 15:00:27.882766 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application [None req-92e22a7f-ce60-4d81-8b6b-0cb934a47cd0 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 23 15:00:27.882766 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:00:27.882766 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:00:27.882766 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:00:27.882766 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:00:27.882766 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:00:27.882766 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:00:27.882766 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:00:27.882766 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:00:27.882766 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:00:27.882766 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:00:27.882766 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:00:27.882766 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 15:00:27.882766 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 15:00:27.882766 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 15:00:27.882766 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 15:00:27.884605 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 15:00:27.884605 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 15:00:27.884605 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 15:00:27.884605 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 15:00:27.884605 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:00:27.884605 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:00:27.884605 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:00:27.884605 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:00:27.884605 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:00:27.884605 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:00:27.884605 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:00:27.884605 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application with Lock( Sep 23 15:00:27.884605 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:00:27.884605 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:00:27.884605 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:00:27.884605 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:00:27.885762 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:00:27.885762 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:00:27.885762 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:00:27.885762 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:00:27.885762 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 15:00:27.885762 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 15:00:27.885762 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 15:00:27.885762 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 15:00:27.885762 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 15:00:27.885762 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 15:00:27.885762 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 23 15:00:27.885762 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Sep 23 15:00:27.885762 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 529/1089] 10.222.0.55 () {56 vars in 1120 bytes} [Fri Sep 23 15:00:27 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:00:27.885762 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e22a7f-ce60-4d81-8b6b-0cb934a47cd0 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:00:27.889475 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d742ead-06d1-40c2-84c6-7b328cc62bb5 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:00:27.924540 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bf871d-6f91-4462-b1a6-2e1201a0be3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:00:27.925180 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bf871d-6f91-4462-b1a6-2e1201a0be3c None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:27.925458 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bf871d-6f91-4462-b1a6-2e1201a0be3c None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:27.925732 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bf871d-6f91-4462-b1a6-2e1201a0be3c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:27.936259 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bf871d-6f91-4462-b1a6-2e1201a0be3c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 15:00:27.941041 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-42bf871d-6f91-4462-b1a6-2e1201a0be3c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 23 15:00:27.941041 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:00:27.941041 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:00:27.941041 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:00:27.941041 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:00:27.941041 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:00:27.941041 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:00:27.941041 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:00:27.941041 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:00:27.941041 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:00:27.941041 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:00:27.941041 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:00:27.941041 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 15:00:27.941041 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 15:00:27.941041 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 15:00:27.941041 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 15:00:27.942070 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 15:00:27.942070 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 15:00:27.942070 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 15:00:27.942070 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 15:00:27.942070 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:00:27.942070 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:00:27.942070 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:00:27.942070 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:00:27.942070 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:00:27.942070 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:00:27.942070 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:00:27.942070 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Sep 23 15:00:27.942070 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:00:27.942070 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:00:27.942070 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:00:27.942070 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:00:27.943120 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:00:27.943120 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:00:27.943120 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:00:27.943120 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:00:27.943120 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 15:00:27.943120 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 15:00:27.943120 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 15:00:27.943120 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 15:00:27.943120 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 15:00:27.943120 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 15:00:27.943120 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 23 15:00:27.943120 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Sep 23 15:00:27.943120 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 561/1090] 10.222.0.55 () {56 vars in 1096 bytes} [Fri Sep 23 15:00:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 23 15:00:27.948720 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bf871d-6f91-4462-b1a6-2e1201a0be3c None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:00:27.972101 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e22a7f-ce60-4d81-8b6b-0cb934a47cd0 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:00:28.006291 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b1a6bd-4211-4dec-8e5e-724c9a443844 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:00:28.006905 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b1a6bd-4211-4dec-8e5e-724c9a443844 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:28.007183 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b1a6bd-4211-4dec-8e5e-724c9a443844 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:28.007457 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b1a6bd-4211-4dec-8e5e-724c9a443844 None admin] PATH_INFO: `/v3/roles` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:28.008154 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b1a6bd-4211-4dec-8e5e-724c9a443844 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:00:28.008461 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b1a6bd-4211-4dec-8e5e-724c9a443844 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:00:28.010452 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b1a6bd-4211-4dec-8e5e-724c9a443844 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:00:28.018506 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 530/1091] 10.222.0.55 () {56 vars in 1115 bytes} [Fri Sep 23 15:00:27 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 15:00:28.024507 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b1a6bd-4211-4dec-8e5e-724c9a443844 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:00:28.028187 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:00:28.049578 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bf871d-6f91-4462-b1a6-2e1201a0be3c None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:00:28.070940 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:00:28.085938 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db09fd7f-4f51-47a8-87be-3b27df44eeee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:00:28.086564 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db09fd7f-4f51-47a8-87be-3b27df44eeee None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:28.086848 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db09fd7f-4f51-47a8-87be-3b27df44eeee None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:28.087148 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db09fd7f-4f51-47a8-87be-3b27df44eeee None admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:28.088375 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db09fd7f-4f51-47a8-87be-3b27df44eeee None admin] RBAC: Adding query filter params (name=admin) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:00:28.088772 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db09fd7f-4f51-47a8-87be-3b27df44eeee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:00:28.090145 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db09fd7f-4f51-47a8-87be-3b27df44eeee None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:00:28.098362 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 562/1092] 10.222.0.55 () {56 vars in 1099 bytes} [Fri Sep 23 15:00:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 15:00:28.104391 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db09fd7f-4f51-47a8-87be-3b27df44eeee None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:00:28.128479 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b1a6bd-4211-4dec-8e5e-724c9a443844 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:00:28.162892 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9783690d-265a-46b5-a218-27eafa4c4391 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:00:28.163536 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9783690d-265a-46b5-a218-27eafa4c4391 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:28.163812 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9783690d-265a-46b5-a218-27eafa4c4391 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:28.164110 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9783690d-265a-46b5-a218-27eafa4c4391 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:28.175699 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9783690d-265a-46b5-a218-27eafa4c4391 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:00:28.177182 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9783690d-265a-46b5-a218-27eafa4c4391 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:00:28.191959 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application [None req-9783690d-265a-46b5-a218-27eafa4c4391 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 15:00:28.191959 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:00:28.191959 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:00:28.191959 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:00:28.191959 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:00:28.191959 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:00:28.191959 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:00:28.191959 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:00:28.191959 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:00:28.191959 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:00:28.191959 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:00:28.191959 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:00:28.191959 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 15:00:28.191959 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 15:00:28.191959 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 15:00:28.191959 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 15:00:28.193004 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:00:28.193004 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:00:28.193004 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:00:28.193004 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:00:28.193004 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:00:28.193004 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:00:28.193004 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:00:28.193004 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application with Lock( Sep 23 15:00:28.193004 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:00:28.193004 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:00:28.193004 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:00:28.193004 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:00:28.193004 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:00:28.193004 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:00:28.193004 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:00:28.193004 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:00:28.193004 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 15:00:28.193952 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 15:00:28.193952 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 15:00:28.193952 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Sep 23 15:00:28.193952 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 531/1093] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Sep 23 15:00:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:00:28.198860 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9783690d-265a-46b5-a218-27eafa4c4391 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:00:28.208488 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db09fd7f-4f51-47a8-87be-3b27df44eeee None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:00:28.244778 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f42c0c-7435-47fb-b46f-b1ae7c1a0c3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:00:28.245697 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f42c0c-7435-47fb-b46f-b1ae7c1a0c3b None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:28.246126 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f42c0c-7435-47fb-b46f-b1ae7c1a0c3b None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:28.246401 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f42c0c-7435-47fb-b46f-b1ae7c1a0c3b None admin] PATH_INFO: `/v3/users/7b2d264e3df84671a429209719cd6b9b` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:28.247537 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f42c0c-7435-47fb-b46f-b1ae7c1a0c3b None admin] RBAC: Authorizing `identity:get_user(user_id=7b2d264e3df84671a429209719cd6b9b)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:00:28.248988 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f42c0c-7435-47fb-b46f-b1ae7c1a0c3b None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:00:28.250917 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 563/1094] 10.222.0.55 () {56 vars in 1177 bytes} [Fri Sep 23 15:00:28 2022] GET /identity/v3/users/7b2d264e3df84671a429209719cd6b9b => generated 285 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 15:00:28.256672 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f42c0c-7435-47fb-b46f-b1ae7c1a0c3b None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:00:28.267692 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:00:28.293731 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:00:28.299364 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9783690d-265a-46b5-a218-27eafa4c4391 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:00:28.333848 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f3ac3c-cca8-456e-8667-0079ba1234bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:00:28.334463 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f3ac3c-cca8-456e-8667-0079ba1234bc None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:28.334744 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f3ac3c-cca8-456e-8667-0079ba1234bc None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:28.335046 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f3ac3c-cca8-456e-8667-0079ba1234bc None admin] PATH_INFO: `/v3/domains` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:28.335691 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f3ac3c-cca8-456e-8667-0079ba1234bc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:00:28.336022 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f3ac3c-cca8-456e-8667-0079ba1234bc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:00:28.337922 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f3ac3c-cca8-456e-8667-0079ba1234bc None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:00:28.349689 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 532/1095] 10.222.0.55 () {56 vars in 1109 bytes} [Fri Sep 23 15:00:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 15:00:28.355947 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f3ac3c-cca8-456e-8667-0079ba1234bc None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:00:28.364351 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f42c0c-7435-47fb-b46f-b1ae7c1a0c3b None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:00:28.398927 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c0a078-23a7-4edb-8956-289bf99c3bb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:00:28.399545 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c0a078-23a7-4edb-8956-289bf99c3bb3 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:28.400397 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c0a078-23a7-4edb-8956-289bf99c3bb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:28.400916 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c0a078-23a7-4edb-8956-289bf99c3bb3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:28.454501 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f3ac3c-cca8-456e-8667-0079ba1234bc None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:00:28.473171 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c0a078-23a7-4edb-8956-289bf99c3bb3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:00:28.476031 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c0a078-23a7-4edb-8956-289bf99c3bb3 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:00:28.488183 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab976f0-50d3-4b66-be00-cae9a75e095b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:00:28.488400 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:00:28.488813 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab976f0-50d3-4b66-be00-cae9a75e095b None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:28.489092 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab976f0-50d3-4b66-be00-cae9a75e095b None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:28.489362 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab976f0-50d3-4b66-be00-cae9a75e095b None admin] PATH_INFO: `/v3/users/nova` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:28.501356 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab976f0-50d3-4b66-be00-cae9a75e095b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:00:28.501356 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab976f0-50d3-4b66-be00-cae9a75e095b None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:00:28.511708 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application [None req-dab976f0-50d3-4b66-be00-cae9a75e095b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 23 15:00:28.511708 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:00:28.511708 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:00:28.511708 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:00:28.511708 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:00:28.511708 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:00:28.511708 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:00:28.511708 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:00:28.511708 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:00:28.511708 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:00:28.511708 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:00:28.511708 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:00:28.511708 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 23 15:00:28.511708 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 23 15:00:28.511708 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 23 15:00:28.511708 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 23 15:00:28.512826 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:00:28.512826 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:00:28.512826 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 23 15:00:28.512826 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 15:00:28.512826 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 23 15:00:28.512826 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 15:00:28.512826 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:00:28.512826 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:00:28.512826 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:00:28.512826 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:00:28.512826 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:00:28.512826 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application with Lock( Sep 23 15:00:28.512826 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:00:28.512826 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:00:28.512826 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:00:28.512826 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:00:28.512826 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:00:28.512826 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:00:28.513795 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:00:28.513795 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:00:28.513795 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 23 15:00:28.513795 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 23 15:00:28.513795 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 23 15:00:28.513795 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 23 15:00:28.513795 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 23 15:00:28.513795 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 23 15:00:28.513795 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 23 15:00:28.513795 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Sep 23 15:00:28.513795 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 533/1096] 10.222.0.55 () {56 vars in 1093 bytes} [Fri Sep 23 15:00:28 2022] GET /identity/v3/users/nova => generated 82 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 23 15:00:28.518571 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab976f0-50d3-4b66-be00-cae9a75e095b None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:00:28.536828 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:00:28.582024 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 564/1097] 10.222.0.55 () {58 vars in 1282 bytes} [Fri Sep 23 15:00:28 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 23 15:00:28.588382 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c0a078-23a7-4edb-8956-289bf99c3bb3 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:00:28.622842 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab976f0-50d3-4b66-be00-cae9a75e095b None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:00:28.656678 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452379bf-8ec5-4f03-9d46-bd0fb1a0cd76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:00:28.657308 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452379bf-8ec5-4f03-9d46-bd0fb1a0cd76 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:28.657609 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452379bf-8ec5-4f03-9d46-bd0fb1a0cd76 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:28.657880 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452379bf-8ec5-4f03-9d46-bd0fb1a0cd76 None admin] PATH_INFO: `/v3/users` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:28.658926 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452379bf-8ec5-4f03-9d46-bd0fb1a0cd76 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:00:28.659230 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452379bf-8ec5-4f03-9d46-bd0fb1a0cd76 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:00:28.660772 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452379bf-8ec5-4f03-9d46-bd0fb1a0cd76 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:00:28.679180 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 534/1098] 10.222.0.55 () {56 vars in 1133 bytes} [Fri Sep 23 15:00:28 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 15:00:28.685746 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452379bf-8ec5-4f03-9d46-bd0fb1a0cd76 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:00:28.689149 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c0a078-23a7-4edb-8956-289bf99c3bb3 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:00:28.698614 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:00:28.723642 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09de363-4748-4874-befc-82f32933ce80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:00:28.724315 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09de363-4748-4874-befc-82f32933ce80 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:28.724601 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09de363-4748-4874-befc-82f32933ce80 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:28.724894 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09de363-4748-4874-befc-82f32933ce80 None admin] PATH_INFO: `/v3/projects/8df0b9101ae94fb697db28a50cd90aeb` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:28.726000 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09de363-4748-4874-befc-82f32933ce80 None admin] RBAC: Authorizing `identity:get_project(project_id=8df0b9101ae94fb697db28a50cd90aeb)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:00:28.727423 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09de363-4748-4874-befc-82f32933ce80 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:00:28.729946 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 565/1099] 10.222.0.55 () {56 vars in 1186 bytes} [Fri Sep 23 15:00:28 2022] GET /identity/v3/projects/8df0b9101ae94fb697db28a50cd90aeb => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 15:00:28.736520 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09de363-4748-4874-befc-82f32933ce80 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:00:28.760912 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:00:28.785380 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452379bf-8ec5-4f03-9d46-bd0fb1a0cd76 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:00:28.826825 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1c7811-2783-40d0-836c-e9ed791bae2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:00:28.827439 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1c7811-2783-40d0-836c-e9ed791bae2b None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:28.827719 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1c7811-2783-40d0-836c-e9ed791bae2b None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:28.828007 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1c7811-2783-40d0-836c-e9ed791bae2b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:28.846451 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09de363-4748-4874-befc-82f32933ce80 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:00:28.885349 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2a4293-bb5c-4d1a-b7c7-7470d3e1863c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:00:28.886024 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2a4293-bb5c-4d1a-b7c7-7470d3e1863c None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:28.886330 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2a4293-bb5c-4d1a-b7c7-7470d3e1863c None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:28.886643 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2a4293-bb5c-4d1a-b7c7-7470d3e1863c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:28.887429 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2a4293-bb5c-4d1a-b7c7-7470d3e1863c None admin] RBAC: Adding query filter params (role.id=8e2919e79df040e8926f21607818b58c, scope.project.id=8df0b9101ae94fb697db28a50cd90aeb, user.id=7b2d264e3df84671a429209719cd6b9b) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:00:28.887758 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2a4293-bb5c-4d1a-b7c7-7470d3e1863c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:00:28.889404 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2a4293-bb5c-4d1a-b7c7-7470d3e1863c None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:00:28.895469 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 566/1100] 10.222.0.55 () {56 vars in 1374 bytes} [Fri Sep 23 15:00:28 2022] GET /identity/v3/role_assignments?user.id=7b2d264e3df84671a429209719cd6b9b&scope.project.id=8df0b9101ae94fb697db28a50cd90aeb&role.id=8e2919e79df040e8926f21607818b58c => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 15:00:28.896499 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1c7811-2783-40d0-836c-e9ed791bae2b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:00:28.898054 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1c7811-2783-40d0-836c-e9ed791bae2b None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:00:28.981142 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:00:29.007614 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 535/1101] 10.222.0.55 () {58 vars in 1282 bytes} [Fri Sep 23 15:00:28 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 23 15:00:29.013913 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2a4293-bb5c-4d1a-b7c7-7470d3e1863c None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:00:29.060423 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:00:29.130384 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2a4293-bb5c-4d1a-b7c7-7470d3e1863c None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:00:29.164493 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b9d69d-c20e-4512-86c6-32ecdc34abe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:00:29.165251 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b9d69d-c20e-4512-86c6-32ecdc34abe4 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:29.165756 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b9d69d-c20e-4512-86c6-32ecdc34abe4 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:29.166032 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b9d69d-c20e-4512-86c6-32ecdc34abe4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:29.176504 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b9d69d-c20e-4512-86c6-32ecdc34abe4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:00:29.177935 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b9d69d-c20e-4512-86c6-32ecdc34abe4 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:00:29.192859 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-28b9d69d-c20e-4512-86c6-32ecdc34abe4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 15:00:29.192859 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:00:29.192859 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:00:29.192859 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:00:29.192859 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:00:29.192859 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:00:29.192859 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:00:29.192859 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:00:29.192859 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:00:29.192859 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:00:29.192859 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:00:29.192859 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:00:29.192859 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 15:00:29.192859 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 15:00:29.192859 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 15:00:29.192859 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 15:00:29.194048 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:00:29.194048 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:00:29.194048 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:00:29.194048 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:00:29.194048 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:00:29.194048 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:00:29.194048 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:00:29.194048 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Sep 23 15:00:29.194048 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:00:29.194048 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:00:29.194048 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:00:29.194048 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:00:29.194048 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:00:29.194048 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:00:29.194048 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:00:29.194048 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:00:29.194048 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 15:00:29.194997 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 15:00:29.194997 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 15:00:29.194997 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Sep 23 15:00:29.194997 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 567/1102] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Sep 23 15:00:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:00:29.200274 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1c7811-2783-40d0-836c-e9ed791bae2b None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:00:29.300633 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1c7811-2783-40d0-836c-e9ed791bae2b None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:00:29.335762 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a69f779-3f64-4a56-8034-1f50b5886cae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:00:29.336408 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a69f779-3f64-4a56-8034-1f50b5886cae None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:29.336687 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a69f779-3f64-4a56-8034-1f50b5886cae None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:29.336961 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a69f779-3f64-4a56-8034-1f50b5886cae None admin] PATH_INFO: `/v3/domains` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:29.337638 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a69f779-3f64-4a56-8034-1f50b5886cae None admin] RBAC: Adding query filter params (name=Default) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:00:29.337988 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a69f779-3f64-4a56-8034-1f50b5886cae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:00:29.339304 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a69f779-3f64-4a56-8034-1f50b5886cae None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:00:29.350547 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 536/1103] 10.222.0.55 () {56 vars in 1109 bytes} [Fri Sep 23 15:00:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:00:29.357374 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b9d69d-c20e-4512-86c6-32ecdc34abe4 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:00:29.449162 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:00:29.457397 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b9d69d-c20e-4512-86c6-32ecdc34abe4 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:00:29.490244 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e4a900-8dcd-422b-9a8d-fa1f3d20933d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:00:29.490872 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e4a900-8dcd-422b-9a8d-fa1f3d20933d None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:29.491152 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e4a900-8dcd-422b-9a8d-fa1f3d20933d None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:29.491424 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e4a900-8dcd-422b-9a8d-fa1f3d20933d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:29.498747 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e4a900-8dcd-422b-9a8d-fa1f3d20933d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:00:29.500247 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e4a900-8dcd-422b-9a8d-fa1f3d20933d None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:00:29.512883 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-53e4a900-8dcd-422b-9a8d-fa1f3d20933d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 15:00:29.512883 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:00:29.512883 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:00:29.512883 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:00:29.512883 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:00:29.512883 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:00:29.512883 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:00:29.512883 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:00:29.512883 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:00:29.512883 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:00:29.512883 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:00:29.512883 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:00:29.512883 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 23 15:00:29.512883 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 23 15:00:29.512883 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 23 15:00:29.519318 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 23 15:00:29.519318 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:00:29.519318 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:00:29.519318 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:00:29.519318 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:00:29.519318 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:00:29.519318 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:00:29.519318 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:00:29.519318 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Sep 23 15:00:29.519318 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:00:29.519318 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:00:29.519318 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:00:29.519318 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:00:29.519318 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:00:29.519318 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:00:29.519318 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:00:29.519318 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:00:29.520465 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 23 15:00:29.520465 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 23 15:00:29.520465 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 23 15:00:29.520465 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 23 15:00:29.520465 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 23 15:00:29.520465 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 23 15:00:29.520465 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 15:00:29.520465 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Sep 23 15:00:29.520465 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 568/1104] 10.222.0.55 () {56 vars in 1111 bytes} [Fri Sep 23 15:00:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:00:29.522406 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a69f779-3f64-4a56-8034-1f50b5886cae None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:00:29.557345 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:00:29.623966 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a69f779-3f64-4a56-8034-1f50b5886cae None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:00:29.659662 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8e9c81-aaab-4062-9631-a484accf10d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:00:29.660306 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8e9c81-aaab-4062-9631-a484accf10d7 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:29.660592 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8e9c81-aaab-4062-9631-a484accf10d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:29.660864 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8e9c81-aaab-4062-9631-a484accf10d7 None admin] PATH_INFO: `/v3/projects` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:29.661552 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8e9c81-aaab-4062-9631-a484accf10d7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:00:29.661859 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8e9c81-aaab-4062-9631-a484accf10d7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:00:29.663284 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8e9c81-aaab-4062-9631-a484accf10d7 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:00:29.675923 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 537/1105] 10.222.0.55 () {56 vars in 1148 bytes} [Fri Sep 23 15:00:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:00:29.682527 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e4a900-8dcd-422b-9a8d-fa1f3d20933d None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:00:29.783510 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e4a900-8dcd-422b-9a8d-fa1f3d20933d None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:00:29.817733 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e340c9-4a8c-48fe-abb6-d128d35c1add None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:00:29.818254 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e340c9-4a8c-48fe-abb6-d128d35c1add None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:29.818534 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e340c9-4a8c-48fe-abb6-d128d35c1add None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:29.818807 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e340c9-4a8c-48fe-abb6-d128d35c1add None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:29.819551 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e340c9-4a8c-48fe-abb6-d128d35c1add None admin] RBAC: Adding query filter params (role.id=f2f7d48ece4f42f682922c1f1beefbd9, scope.project.id=87b3e9d4b6474bd4a04ba59ef52bf3e2, user.id=b3c6c7a1325a47be924268e1aa1a7a18) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:00:29.819861 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e340c9-4a8c-48fe-abb6-d128d35c1add None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:00:29.821360 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e340c9-4a8c-48fe-abb6-d128d35c1add None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:00:29.828589 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 569/1106] 10.222.0.55 () {56 vars in 1374 bytes} [Fri Sep 23 15:00:29 2022] GET /identity/v3/role_assignments?user.id=b3c6c7a1325a47be924268e1aa1a7a18&scope.project.id=87b3e9d4b6474bd4a04ba59ef52bf3e2&role.id=f2f7d48ece4f42f682922c1f1beefbd9 => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 15:00:31.128376 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34b12d1-1069-442f-82ad-dbc8c7ac16a0 None None] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:31.128916 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34b12d1-1069-442f-82ad-dbc8c7ac16a0 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:31.128916 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34b12d1-1069-442f-82ad-dbc8c7ac16a0 None None] PATH_INFO: `/` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:31.130249 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 538/1107] 10.222.0.55 () {54 vars in 927 bytes} [Fri Sep 23 15:00:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:00:31.136974 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2490d30c-cf16-4821-b253-fe41e30083af None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:31.137388 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2490d30c-cf16-4821-b253-fe41e30083af None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:31.137535 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2490d30c-cf16-4821-b253-fe41e30083af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:31.156192 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-2490d30c-cf16-4821-b253-fe41e30083af None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:00:31.157376 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-2490d30c-cf16-4821-b253-fe41e30083af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:00:31.168757 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:00:31.224847 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 570/1108] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Sep 23 15:00:31 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 23 15:00:31.503170 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34b12d1-1069-442f-82ad-dbc8c7ac16a0 None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:00:31.597009 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:00:31.605417 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34b12d1-1069-442f-82ad-dbc8c7ac16a0 None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:00:31.641436 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d499673a-7b38-4546-963c-97358f313c2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:00:31.642103 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d499673a-7b38-4546-963c-97358f313c2b None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:31.642378 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d499673a-7b38-4546-963c-97358f313c2b None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:31.642682 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d499673a-7b38-4546-963c-97358f313c2b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:31.643854 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d499673a-7b38-4546-963c-97358f313c2b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:00:31.645449 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d499673a-7b38-4546-963c-97358f313c2b None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:00:31.647103 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 539/1109] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Sep 23 15:00:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:00:31.653597 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2490d30c-cf16-4821-b253-fe41e30083af None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:00:31.758615 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2490d30c-cf16-4821-b253-fe41e30083af None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:00:31.798601 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbaa61e-c925-4dd3-8ed1-45b6ccaa5a5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:00:31.799245 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbaa61e-c925-4dd3-8ed1-45b6ccaa5a5a None admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:31.799900 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbaa61e-c925-4dd3-8ed1-45b6ccaa5a5a None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:31.800208 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbaa61e-c925-4dd3-8ed1-45b6ccaa5a5a None admin] PATH_INFO: `/v3/users` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:31.800882 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbaa61e-c925-4dd3-8ed1-45b6ccaa5a5a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:00:31.802340 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbaa61e-c925-4dd3-8ed1-45b6ccaa5a5a None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:00:31.933113 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 571/1110] 10.222.0.55 () {60 vars in 1132 bytes} [Fri Sep 23 15:00:31 2022] POST /identity/v3/users => generated 286 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:00:34.287881 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003e4533-a8a3-4d7f-b5be-6ca372fdb0f8 None None] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:34.288439 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003e4533-a8a3-4d7f-b5be-6ca372fdb0f8 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:34.288526 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003e4533-a8a3-4d7f-b5be-6ca372fdb0f8 None None] PATH_INFO: `/` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:34.289784 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 540/1111] 10.222.0.55 () {54 vars in 927 bytes} [Fri Sep 23 15:00:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:00:34.295917 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa404290-5ab3-4317-8043-47f61a21f114 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:34.296349 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa404290-5ab3-4317-8043-47f61a21f114 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:34.296550 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa404290-5ab3-4317-8043-47f61a21f114 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:34.302901 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:00:34.315426 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-fa404290-5ab3-4317-8043-47f61a21f114 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:00:34.316692 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-fa404290-5ab3-4317-8043-47f61a21f114 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:00:34.385690 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 572/1112] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Sep 23 15:00:34 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 23 15:00:34.663643 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003e4533-a8a3-4d7f-b5be-6ca372fdb0f8 None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:00:34.763523 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003e4533-a8a3-4d7f-b5be-6ca372fdb0f8 None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:00:34.796519 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b731ed-2120-44c3-857a-572e715ef8d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:00:34.797141 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b731ed-2120-44c3-857a-572e715ef8d2 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:34.797419 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b731ed-2120-44c3-857a-572e715ef8d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:34.797691 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b731ed-2120-44c3-857a-572e715ef8d2 None admin] PATH_INFO: `/v3/roles/5c74fdc0e9cc44538cd8282006ed4443` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:34.799067 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b731ed-2120-44c3-857a-572e715ef8d2 None admin] RBAC: Authorizing `identity:get_role(role_id=5c74fdc0e9cc44538cd8282006ed4443)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:00:34.800459 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b731ed-2120-44c3-857a-572e715ef8d2 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:00:34.801758 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 541/1113] 10.222.0.55 () {56 vars in 1177 bytes} [Fri Sep 23 15:00:34 2022] GET /identity/v3/roles/5c74fdc0e9cc44538cd8282006ed4443 => generated 223 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:00:34.808812 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa404290-5ab3-4317-8043-47f61a21f114 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:00:34.909362 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa404290-5ab3-4317-8043-47f61a21f114 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:00:34.939133 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:00:34.944024 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2485862b-5ffb-4fc8-8dea-b8135b016d48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:00:34.944653 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2485862b-5ffb-4fc8-8dea-b8135b016d48 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:34.944932 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2485862b-5ffb-4fc8-8dea-b8135b016d48 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:34.945204 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2485862b-5ffb-4fc8-8dea-b8135b016d48 None admin] PATH_INFO: `/v3/users/1b5f58362cbc44b4bc8a1d492324569c` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:34.958626 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2485862b-5ffb-4fc8-8dea-b8135b016d48 None admin] RBAC: Authorizing `identity:get_user(user_id=1b5f58362cbc44b4bc8a1d492324569c)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:00:34.960555 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2485862b-5ffb-4fc8-8dea-b8135b016d48 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:00:34.961761 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 573/1114] 10.222.0.55 () {56 vars in 1177 bytes} [Fri Sep 23 15:00:34 2022] GET /identity/v3/users/1b5f58362cbc44b4bc8a1d492324569c => generated 286 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:00:34.968828 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b731ed-2120-44c3-857a-572e715ef8d2 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:00:35.000336 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:00:35.074562 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b731ed-2120-44c3-857a-572e715ef8d2 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:00:35.109111 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc99e36-73df-4dcf-98f9-4a2a4b8f73db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:00:35.109720 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc99e36-73df-4dcf-98f9-4a2a4b8f73db None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:35.110001 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc99e36-73df-4dcf-98f9-4a2a4b8f73db None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:35.110294 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc99e36-73df-4dcf-98f9-4a2a4b8f73db None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:35.176455 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc99e36-73df-4dcf-98f9-4a2a4b8f73db None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:00:35.177881 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc99e36-73df-4dcf-98f9-4a2a4b8f73db None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:00:35.206550 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:00:35.273998 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 542/1115] 10.222.0.55 () {58 vars in 1282 bytes} [Fri Sep 23 15:00:34 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 23 15:00:35.280132 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2485862b-5ffb-4fc8-8dea-b8135b016d48 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:00:35.381532 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2485862b-5ffb-4fc8-8dea-b8135b016d48 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:00:35.417344 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def0c6d4-8080-4553-8830-eb8a6a9569f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:00:35.417966 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def0c6d4-8080-4553-8830-eb8a6a9569f4 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:35.418249 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def0c6d4-8080-4553-8830-eb8a6a9569f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:35.418523 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def0c6d4-8080-4553-8830-eb8a6a9569f4 None admin] PATH_INFO: `/v3/projects/8df0b9101ae94fb697db28a50cd90aeb` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:35.419645 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def0c6d4-8080-4553-8830-eb8a6a9569f4 None admin] RBAC: Authorizing `identity:get_project(project_id=8df0b9101ae94fb697db28a50cd90aeb)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:00:35.421137 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def0c6d4-8080-4553-8830-eb8a6a9569f4 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:00:35.422757 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 574/1116] 10.222.0.55 () {56 vars in 1186 bytes} [Fri Sep 23 15:00:35 2022] GET /identity/v3/projects/8df0b9101ae94fb697db28a50cd90aeb => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:00:35.429126 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc99e36-73df-4dcf-98f9-4a2a4b8f73db None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:00:35.529276 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc99e36-73df-4dcf-98f9-4a2a4b8f73db None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:00:35.559073 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:00:35.564851 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0219db2d-bca6-44f1-a087-cc0c49dd6746 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:00:35.565454 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0219db2d-bca6-44f1-a087-cc0c49dd6746 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:35.565731 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0219db2d-bca6-44f1-a087-cc0c49dd6746 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:35.566001 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0219db2d-bca6-44f1-a087-cc0c49dd6746 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:35.566717 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0219db2d-bca6-44f1-a087-cc0c49dd6746 None admin] RBAC: Adding query filter params (role.id=5c74fdc0e9cc44538cd8282006ed4443, scope.project.id=8df0b9101ae94fb697db28a50cd90aeb, user.id=1b5f58362cbc44b4bc8a1d492324569c) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:00:35.567018 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0219db2d-bca6-44f1-a087-cc0c49dd6746 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:00:35.568471 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0219db2d-bca6-44f1-a087-cc0c49dd6746 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:00:35.573879 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 543/1117] 10.222.0.55 () {56 vars in 1374 bytes} [Fri Sep 23 15:00:35 2022] GET /identity/v3/role_assignments?user.id=1b5f58362cbc44b4bc8a1d492324569c&scope.project.id=8df0b9101ae94fb697db28a50cd90aeb&role.id=5c74fdc0e9cc44538cd8282006ed4443 => generated 259 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:00:37.761690 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a306d29-4186-4e38-b741-ad46280aad29 None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:37.762260 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a306d29-4186-4e38-b741-ad46280aad29 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:37.762260 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a306d29-4186-4e38-b741-ad46280aad29 None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:37.763569 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 575/1118] 10.222.0.55 () {54 vars in 927 bytes} [Fri Sep 23 15:00:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:00:37.768908 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef21ce87-12a5-4572-9cce-de3fae1f8ec0 None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:37.769306 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef21ce87-12a5-4572-9cce-de3fae1f8ec0 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:37.769485 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef21ce87-12a5-4572-9cce-de3fae1f8ec0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:37.792791 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-ef21ce87-12a5-4572-9cce-de3fae1f8ec0 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:00:37.794005 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-ef21ce87-12a5-4572-9cce-de3fae1f8ec0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:00:37.869831 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 544/1119] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Sep 23 15:00:37 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 23 15:00:38.143427 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a306d29-4186-4e38-b741-ad46280aad29 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:00:38.204276 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:00:38.244898 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a306d29-4186-4e38-b741-ad46280aad29 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:00:38.278573 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5713bba2-cda6-48d2-bfea-3439410c3244 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:00:38.279205 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5713bba2-cda6-48d2-bfea-3439410c3244 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:38.279487 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5713bba2-cda6-48d2-bfea-3439410c3244 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:38.279763 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5713bba2-cda6-48d2-bfea-3439410c3244 None admin] PATH_INFO: `/v3/roles/5c74fdc0e9cc44538cd8282006ed4443` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:38.281184 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5713bba2-cda6-48d2-bfea-3439410c3244 None admin] RBAC: Authorizing `identity:get_role(role_id=5c74fdc0e9cc44538cd8282006ed4443)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:00:38.282543 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5713bba2-cda6-48d2-bfea-3439410c3244 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:00:38.284155 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 576/1120] 10.222.0.55 () {56 vars in 1177 bytes} [Fri Sep 23 15:00:38 2022] GET /identity/v3/roles/5c74fdc0e9cc44538cd8282006ed4443 => generated 223 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:00:38.290195 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef21ce87-12a5-4572-9cce-de3fae1f8ec0 None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:00:38.387922 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef21ce87-12a5-4572-9cce-de3fae1f8ec0 None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:00:38.411034 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:00:38.422039 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8445b119-1bfb-4f2c-9077-3fa5b003607b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:00:38.422671 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8445b119-1bfb-4f2c-9077-3fa5b003607b None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:38.424782 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8445b119-1bfb-4f2c-9077-3fa5b003607b None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:38.424782 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8445b119-1bfb-4f2c-9077-3fa5b003607b None admin] PATH_INFO: `/v3/users/1b5f58362cbc44b4bc8a1d492324569c` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:38.424782 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8445b119-1bfb-4f2c-9077-3fa5b003607b None admin] RBAC: Authorizing `identity:get_user(user_id=1b5f58362cbc44b4bc8a1d492324569c)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:00:38.425759 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8445b119-1bfb-4f2c-9077-3fa5b003607b None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:00:38.427308 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 545/1121] 10.222.0.55 () {56 vars in 1177 bytes} [Fri Sep 23 15:00:38 2022] GET /identity/v3/users/1b5f58362cbc44b4bc8a1d492324569c => generated 286 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:00:38.433943 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5713bba2-cda6-48d2-bfea-3439410c3244 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:00:38.530407 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5713bba2-cda6-48d2-bfea-3439410c3244 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:00:38.555614 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:00:38.564576 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec87e611-3fbc-4dde-b8a8-64a566adb8c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:00:38.565207 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec87e611-3fbc-4dde-b8a8-64a566adb8c1 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:38.567755 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec87e611-3fbc-4dde-b8a8-64a566adb8c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:38.567755 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec87e611-3fbc-4dde-b8a8-64a566adb8c1 None admin] PATH_INFO: `/v3/projects/8df0b9101ae94fb697db28a50cd90aeb` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:38.567755 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec87e611-3fbc-4dde-b8a8-64a566adb8c1 None admin] RBAC: Authorizing `identity:get_project(project_id=8df0b9101ae94fb697db28a50cd90aeb)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:00:38.568272 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec87e611-3fbc-4dde-b8a8-64a566adb8c1 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:00:38.569880 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 577/1122] 10.222.0.55 () {56 vars in 1186 bytes} [Fri Sep 23 15:00:38 2022] GET /identity/v3/projects/8df0b9101ae94fb697db28a50cd90aeb => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:00:38.576101 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8445b119-1bfb-4f2c-9077-3fa5b003607b None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:00:38.673862 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8445b119-1bfb-4f2c-9077-3fa5b003607b None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:00:38.707522 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c1c6c4-7c43-4dd3-8677-0a5b16279f19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:00:38.708200 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c1c6c4-7c43-4dd3-8677-0a5b16279f19 None admin] REQUEST_METHOD: `PUT` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:38.708480 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c1c6c4-7c43-4dd3-8677-0a5b16279f19 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:38.708751 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c1c6c4-7c43-4dd3-8677-0a5b16279f19 None admin] PATH_INFO: `/v3/projects/8df0b9101ae94fb697db28a50cd90aeb/users/1b5f58362cbc44b4bc8a1d492324569c/roles/5c74fdc0e9cc44538cd8282006ed4443` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:38.710864 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c1c6c4-7c43-4dd3-8677-0a5b16279f19 None admin] RBAC: Authorizing `identity:create_grant(project_id=8df0b9101ae94fb697db28a50cd90aeb, user_id=1b5f58362cbc44b4bc8a1d492324569c, role_id=5c74fdc0e9cc44538cd8282006ed4443)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:00:38.712923 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c1c6c4-7c43-4dd3-8677-0a5b16279f19 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:00:38.806620 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 546/1123] 10.222.0.55 () {58 vars in 1439 bytes} [Fri Sep 23 15:00:38 2022] PUT /identity/v3/projects/8df0b9101ae94fb697db28a50cd90aeb/users/1b5f58362cbc44b4bc8a1d492324569c/roles/5c74fdc0e9cc44538cd8282006ed4443 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:00:41.002342 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6cac05-4a19-432f-b7dd-dd197334a05a None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:41.002879 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6cac05-4a19-432f-b7dd-dd197334a05a None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:41.002879 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6cac05-4a19-432f-b7dd-dd197334a05a None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:41.004237 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 578/1124] 10.222.0.55 () {54 vars in 927 bytes} [Fri Sep 23 15:00:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:00:41.010342 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c137eeb-c104-4276-9e6a-7214a64e78ab None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:41.010759 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c137eeb-c104-4276-9e6a-7214a64e78ab None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:41.010871 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c137eeb-c104-4276-9e6a-7214a64e78ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:41.029201 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-7c137eeb-c104-4276-9e6a-7214a64e78ab None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:00:41.030386 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-7c137eeb-c104-4276-9e6a-7214a64e78ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:00:41.078519 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:00:41.098135 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 547/1125] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Sep 23 15:00:41 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 23 15:00:41.368365 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6cac05-4a19-432f-b7dd-dd197334a05a None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:00:41.471446 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6cac05-4a19-432f-b7dd-dd197334a05a None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:00:41.506275 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941de27c-2f00-4348-bcfa-332b64c3448e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:00:41.506926 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941de27c-2f00-4348-bcfa-332b64c3448e None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:41.507214 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941de27c-2f00-4348-bcfa-332b64c3448e None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:41.507547 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941de27c-2f00-4348-bcfa-332b64c3448e None admin] PATH_INFO: `/v3/roles/5c74fdc0e9cc44538cd8282006ed4443` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:41.509008 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941de27c-2f00-4348-bcfa-332b64c3448e None admin] RBAC: Authorizing `identity:get_role(role_id=5c74fdc0e9cc44538cd8282006ed4443)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:00:41.510755 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941de27c-2f00-4348-bcfa-332b64c3448e None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:00:41.512156 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 579/1126] 10.222.0.55 () {56 vars in 1177 bytes} [Fri Sep 23 15:00:41 2022] GET /identity/v3/roles/5c74fdc0e9cc44538cd8282006ed4443 => generated 223 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:00:41.517943 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c137eeb-c104-4276-9e6a-7214a64e78ab None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:00:41.614757 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c137eeb-c104-4276-9e6a-7214a64e78ab None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:00:41.648809 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac31dee-a40f-4353-8c73-6ec701a32568 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:00:41.649415 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac31dee-a40f-4353-8c73-6ec701a32568 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:41.649703 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac31dee-a40f-4353-8c73-6ec701a32568 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:41.649973 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac31dee-a40f-4353-8c73-6ec701a32568 None admin] PATH_INFO: `/v3/users/1b5f58362cbc44b4bc8a1d492324569c` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:41.651079 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac31dee-a40f-4353-8c73-6ec701a32568 None admin] RBAC: Authorizing `identity:get_user(user_id=1b5f58362cbc44b4bc8a1d492324569c)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:00:41.652534 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac31dee-a40f-4353-8c73-6ec701a32568 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:00:41.654088 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 548/1127] 10.222.0.55 () {56 vars in 1177 bytes} [Fri Sep 23 15:00:41 2022] GET /identity/v3/users/1b5f58362cbc44b4bc8a1d492324569c => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:00:41.660197 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941de27c-2f00-4348-bcfa-332b64c3448e None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:00:41.720092 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:00:41.756503 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941de27c-2f00-4348-bcfa-332b64c3448e None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:00:41.792986 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ee29c7-6e91-41c3-aa3b-56bd5b9db984 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:00:41.793591 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ee29c7-6e91-41c3-aa3b-56bd5b9db984 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:41.793866 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ee29c7-6e91-41c3-aa3b-56bd5b9db984 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:41.794132 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ee29c7-6e91-41c3-aa3b-56bd5b9db984 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:41.859038 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ee29c7-6e91-41c3-aa3b-56bd5b9db984 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:00:41.860499 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ee29c7-6e91-41c3-aa3b-56bd5b9db984 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:00:41.920354 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:00:41.958373 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 580/1128] 10.222.0.55 () {58 vars in 1282 bytes} [Fri Sep 23 15:00:41 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 23 15:00:41.964977 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac31dee-a40f-4353-8c73-6ec701a32568 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:00:42.019938 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:00:42.069897 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac31dee-a40f-4353-8c73-6ec701a32568 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:00:42.105826 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8ba5cd-b7fc-4257-942c-ce8510492707 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:00:42.106453 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8ba5cd-b7fc-4257-942c-ce8510492707 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:42.106736 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8ba5cd-b7fc-4257-942c-ce8510492707 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:42.107011 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8ba5cd-b7fc-4257-942c-ce8510492707 None admin] PATH_INFO: `/v3/projects/8df0b9101ae94fb697db28a50cd90aeb` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:42.108172 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8ba5cd-b7fc-4257-942c-ce8510492707 None admin] RBAC: Authorizing `identity:get_project(project_id=8df0b9101ae94fb697db28a50cd90aeb)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:00:42.109599 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8ba5cd-b7fc-4257-942c-ce8510492707 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:00:42.111203 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 549/1129] 10.222.0.55 () {56 vars in 1186 bytes} [Fri Sep 23 15:00:41 2022] GET /identity/v3/projects/8df0b9101ae94fb697db28a50cd90aeb => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:00:42.117435 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ee29c7-6e91-41c3-aa3b-56bd5b9db984 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:00:42.216622 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ee29c7-6e91-41c3-aa3b-56bd5b9db984 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:00:42.249535 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d80d5bb-6d90-41fb-87ba-cb2bc0962a28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:00:42.250140 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d80d5bb-6d90-41fb-87ba-cb2bc0962a28 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:42.250426 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d80d5bb-6d90-41fb-87ba-cb2bc0962a28 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:42.250698 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d80d5bb-6d90-41fb-87ba-cb2bc0962a28 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:42.251415 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d80d5bb-6d90-41fb-87ba-cb2bc0962a28 None admin] RBAC: Adding query filter params (role.id=5c74fdc0e9cc44538cd8282006ed4443, scope.project.id=8df0b9101ae94fb697db28a50cd90aeb, user.id=1b5f58362cbc44b4bc8a1d492324569c) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:00:42.251723 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d80d5bb-6d90-41fb-87ba-cb2bc0962a28 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:00:42.253177 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d80d5bb-6d90-41fb-87ba-cb2bc0962a28 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:00:42.258794 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 581/1130] 10.222.0.55 () {56 vars in 1374 bytes} [Fri Sep 23 15:00:42 2022] GET /identity/v3/role_assignments?user.id=1b5f58362cbc44b4bc8a1d492324569c&scope.project.id=8df0b9101ae94fb697db28a50cd90aeb&role.id=5c74fdc0e9cc44538cd8282006ed4443 => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:00:44.438937 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84c4421-53be-4d35-b491-2d18bba3c8ac None None] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:44.439457 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84c4421-53be-4d35-b491-2d18bba3c8ac None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:44.439457 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84c4421-53be-4d35-b491-2d18bba3c8ac None None] PATH_INFO: `/` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:44.440748 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 550/1131] 10.222.0.55 () {54 vars in 927 bytes} [Fri Sep 23 15:00:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:00:44.445962 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adeaac54-0ae7-480f-a73b-16c4c0a8cdc2 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:44.446348 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adeaac54-0ae7-480f-a73b-16c4c0a8cdc2 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:44.446523 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adeaac54-0ae7-480f-a73b-16c4c0a8cdc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:44.466391 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-adeaac54-0ae7-480f-a73b-16c4c0a8cdc2 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:00:44.468256 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-adeaac54-0ae7-480f-a73b-16c4c0a8cdc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:00:44.488552 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:00:44.534768 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 582/1132] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Sep 23 15:00:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 23 15:00:44.804336 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84c4421-53be-4d35-b491-2d18bba3c8ac None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:00:44.905909 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84c4421-53be-4d35-b491-2d18bba3c8ac None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:00:44.923576 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:00:44.940280 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f255f3-fdd0-43f3-9a65-2d5c7d5aaa74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:00:44.940910 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f255f3-fdd0-43f3-9a65-2d5c7d5aaa74 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:44.941192 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f255f3-fdd0-43f3-9a65-2d5c7d5aaa74 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:44.941465 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f255f3-fdd0-43f3-9a65-2d5c7d5aaa74 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:44.942574 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f255f3-fdd0-43f3-9a65-2d5c7d5aaa74 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:00:44.944015 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f255f3-fdd0-43f3-9a65-2d5c7d5aaa74 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:00:44.945612 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 551/1133] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Sep 23 15:00:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:00:44.952764 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adeaac54-0ae7-480f-a73b-16c4c0a8cdc2 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:00:45.053547 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adeaac54-0ae7-480f-a73b-16c4c0a8cdc2 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:00:45.091903 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84fd9f1-917d-4359-92d3-65bd2d502b54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:00:45.092550 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84fd9f1-917d-4359-92d3-65bd2d502b54 None admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:45.092832 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84fd9f1-917d-4359-92d3-65bd2d502b54 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:45.093108 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84fd9f1-917d-4359-92d3-65bd2d502b54 None admin] PATH_INFO: `/v3/projects` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:45.093765 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84fd9f1-917d-4359-92d3-65bd2d502b54 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:00:45.095238 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84fd9f1-917d-4359-92d3-65bd2d502b54 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:00:45.247381 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 583/1134] 10.222.0.55 () {60 vars in 1141 bytes} [Fri Sep 23 15:00:44 2022] POST /identity/v3/projects => generated 311 bytes in 295 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:00:47.435348 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738a7769-01cf-4763-bb6a-0d8e07378033 None None] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:47.435866 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738a7769-01cf-4763-bb6a-0d8e07378033 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:47.435866 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738a7769-01cf-4763-bb6a-0d8e07378033 None None] PATH_INFO: `/` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:47.437159 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 552/1135] 10.222.0.55 () {54 vars in 927 bytes} [Fri Sep 23 15:00:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:00:47.442434 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31c89fc-7bbd-4091-8028-a2844653f5d6 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:47.442835 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31c89fc-7bbd-4091-8028-a2844653f5d6 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:47.442975 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31c89fc-7bbd-4091-8028-a2844653f5d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:47.461068 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-d31c89fc-7bbd-4091-8028-a2844653f5d6 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:00:47.462278 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-d31c89fc-7bbd-4091-8028-a2844653f5d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:00:47.464336 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:00:47.528253 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 584/1136] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Sep 23 15:00:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 23 15:00:47.796740 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738a7769-01cf-4763-bb6a-0d8e07378033 None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:00:47.896926 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738a7769-01cf-4763-bb6a-0d8e07378033 None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:00:47.931217 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76702690-e83f-4733-9b09-9f101274a09f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:00:47.932070 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76702690-e83f-4733-9b09-9f101274a09f None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:47.932353 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76702690-e83f-4733-9b09-9f101274a09f None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:47.932625 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76702690-e83f-4733-9b09-9f101274a09f None admin] PATH_INFO: `/v3/domains/default` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:47.933731 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76702690-e83f-4733-9b09-9f101274a09f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:00:47.935153 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76702690-e83f-4733-9b09-9f101274a09f None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:00:47.936728 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 553/1137] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Sep 23 15:00:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:00:47.942689 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31c89fc-7bbd-4091-8028-a2844653f5d6 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:00:48.043188 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31c89fc-7bbd-4091-8028-a2844653f5d6 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:00:48.076830 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22db1b89-9473-4ac8-9e20-c91681b50466 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:00:48.077391 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22db1b89-9473-4ac8-9e20-c91681b50466 None admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:48.077678 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22db1b89-9473-4ac8-9e20-c91681b50466 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:48.077955 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22db1b89-9473-4ac8-9e20-c91681b50466 None admin] PATH_INFO: `/v3/users` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:48.078619 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22db1b89-9473-4ac8-9e20-c91681b50466 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:00:48.080683 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22db1b89-9473-4ac8-9e20-c91681b50466 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:00:48.091465 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:00:48.230144 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 585/1138] 10.222.0.55 () {60 vars in 1132 bytes} [Fri Sep 23 15:00:47 2022] POST /identity/v3/users => generated 286 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 23 15:00:50.422450 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce3913b-cb52-491c-9cd9-4261e7204613 None None] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:50.423011 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce3913b-cb52-491c-9cd9-4261e7204613 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:50.423011 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce3913b-cb52-491c-9cd9-4261e7204613 None None] PATH_INFO: `/` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:50.424455 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 554/1139] 10.222.0.55 () {54 vars in 927 bytes} [Fri Sep 23 15:00:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:00:50.429541 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771f9ee2-25de-4779-97c0-f724f91eb3c2 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:50.429933 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771f9ee2-25de-4779-97c0-f724f91eb3c2 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:50.430090 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771f9ee2-25de-4779-97c0-f724f91eb3c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:50.448553 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-771f9ee2-25de-4779-97c0-f724f91eb3c2 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:00:50.449730 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-771f9ee2-25de-4779-97c0-f724f91eb3c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:00:50.514399 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 586/1140] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Sep 23 15:00:50 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 23 15:00:50.782129 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce3913b-cb52-491c-9cd9-4261e7204613 None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:00:50.838641 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:00:50.881804 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce3913b-cb52-491c-9cd9-4261e7204613 None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:00:50.917351 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d619a55d-7212-4457-a1f7-17016e6f69ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:00:50.917980 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d619a55d-7212-4457-a1f7-17016e6f69ea None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:50.918255 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d619a55d-7212-4457-a1f7-17016e6f69ea None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:50.918525 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d619a55d-7212-4457-a1f7-17016e6f69ea None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:50.928502 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d619a55d-7212-4457-a1f7-17016e6f69ea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 15:00:50.933422 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application [None req-d619a55d-7212-4457-a1f7-17016e6f69ea None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 23 15:00:50.933422 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:00:50.933422 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:00:50.933422 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:00:50.933422 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:00:50.933422 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:00:50.933422 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:00:50.933422 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:00:50.933422 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:00:50.933422 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:00:50.933422 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:00:50.933422 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:00:50.933422 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 15:00:50.933422 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 15:00:50.933422 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 15:00:50.933422 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 15:00:50.934627 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 15:00:50.934627 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 15:00:50.934627 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 15:00:50.934627 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 15:00:50.934627 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:00:50.934627 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:00:50.934627 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:00:50.934627 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:00:50.934627 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:00:50.934627 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:00:50.934627 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:00:50.934627 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application with Lock( Sep 23 15:00:50.934627 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:00:50.934627 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:00:50.934627 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:00:50.934627 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:00:50.935836 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:00:50.935836 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:00:50.935836 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:00:50.935836 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:00:50.935836 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 15:00:50.935836 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 15:00:50.935836 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 15:00:50.935836 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 15:00:50.935836 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 15:00:50.935836 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 15:00:50.935836 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 23 15:00:50.935836 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Sep 23 15:00:50.935836 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 555/1141] 10.222.0.55 () {56 vars in 1096 bytes} [Fri Sep 23 15:00:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:00:50.940967 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771f9ee2-25de-4779-97c0-f724f91eb3c2 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:00:51.117422 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771f9ee2-25de-4779-97c0-f724f91eb3c2 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:00:51.142217 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:00:51.153202 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f167337-dfb1-4422-bd04-1fda623501c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:00:51.153818 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f167337-dfb1-4422-bd04-1fda623501c2 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:51.154122 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f167337-dfb1-4422-bd04-1fda623501c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:51.154400 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f167337-dfb1-4422-bd04-1fda623501c2 None admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:51.155059 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f167337-dfb1-4422-bd04-1fda623501c2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:00:51.155366 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f167337-dfb1-4422-bd04-1fda623501c2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:00:51.156754 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f167337-dfb1-4422-bd04-1fda623501c2 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:00:51.165115 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 587/1142] 10.222.0.55 () {56 vars in 1099 bytes} [Fri Sep 23 15:00:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:00:51.171302 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d619a55d-7212-4457-a1f7-17016e6f69ea None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:00:51.270345 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d619a55d-7212-4457-a1f7-17016e6f69ea None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:00:51.284301 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:00:51.303596 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcffe5d9-8fe3-4775-86e4-89383d89fa40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:00:51.304254 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcffe5d9-8fe3-4775-86e4-89383d89fa40 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:51.304530 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcffe5d9-8fe3-4775-86e4-89383d89fa40 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:51.304800 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcffe5d9-8fe3-4775-86e4-89383d89fa40 None admin] PATH_INFO: `/v3/users/8fe3d6d737b4427081967f517696d2be` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:51.318110 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcffe5d9-8fe3-4775-86e4-89383d89fa40 None admin] RBAC: Authorizing `identity:get_user(user_id=8fe3d6d737b4427081967f517696d2be)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:00:51.319661 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcffe5d9-8fe3-4775-86e4-89383d89fa40 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:00:51.322566 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 556/1143] 10.222.0.55 () {56 vars in 1177 bytes} [Fri Sep 23 15:00:51 2022] GET /identity/v3/users/8fe3d6d737b4427081967f517696d2be => generated 286 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:00:51.328709 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f167337-dfb1-4422-bd04-1fda623501c2 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:00:51.426650 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f167337-dfb1-4422-bd04-1fda623501c2 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:00:51.459827 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40739566-aa8a-4051-b019-d2cf69b87b14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:00:51.460463 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40739566-aa8a-4051-b019-d2cf69b87b14 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:51.460741 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40739566-aa8a-4051-b019-d2cf69b87b14 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:51.461011 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40739566-aa8a-4051-b019-d2cf69b87b14 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:51.514771 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:00:51.527176 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40739566-aa8a-4051-b019-d2cf69b87b14 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:00:51.530325 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40739566-aa8a-4051-b019-d2cf69b87b14 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:00:51.627284 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 588/1144] 10.222.0.55 () {58 vars in 1282 bytes} [Fri Sep 23 15:00:51 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 23 15:00:51.633447 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcffe5d9-8fe3-4775-86e4-89383d89fa40 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:00:51.736345 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcffe5d9-8fe3-4775-86e4-89383d89fa40 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:00:51.772394 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c89a19f-78f8-4ce0-9897-f22c693cbfac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:00:51.773029 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c89a19f-78f8-4ce0-9897-f22c693cbfac None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:51.773308 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c89a19f-78f8-4ce0-9897-f22c693cbfac None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:51.773582 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c89a19f-78f8-4ce0-9897-f22c693cbfac None admin] PATH_INFO: `/v3/projects/8b33b96fe121426d87db440adfed911c` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:51.774684 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c89a19f-78f8-4ce0-9897-f22c693cbfac None admin] RBAC: Authorizing `identity:get_project(project_id=8b33b96fe121426d87db440adfed911c)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:00:51.776663 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c89a19f-78f8-4ce0-9897-f22c693cbfac None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:00:51.778284 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 557/1145] 10.222.0.55 () {56 vars in 1186 bytes} [Fri Sep 23 15:00:51 2022] GET /identity/v3/projects/8b33b96fe121426d87db440adfed911c => generated 311 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:00:51.784792 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40739566-aa8a-4051-b019-d2cf69b87b14 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:00:51.869057 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:00:51.883479 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40739566-aa8a-4051-b019-d2cf69b87b14 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:00:51.918877 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2983e4f8-724c-4fd6-b277-22e0ddd1e18c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:00:51.919538 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2983e4f8-724c-4fd6-b277-22e0ddd1e18c None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:51.919843 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2983e4f8-724c-4fd6-b277-22e0ddd1e18c None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:51.920151 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2983e4f8-724c-4fd6-b277-22e0ddd1e18c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:51.920873 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2983e4f8-724c-4fd6-b277-22e0ddd1e18c None admin] RBAC: Adding query filter params (role.id=8e2919e79df040e8926f21607818b58c, scope.project.id=8b33b96fe121426d87db440adfed911c, user.id=8fe3d6d737b4427081967f517696d2be) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:00:51.921183 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2983e4f8-724c-4fd6-b277-22e0ddd1e18c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:00:51.922606 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2983e4f8-724c-4fd6-b277-22e0ddd1e18c None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:00:51.928383 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 589/1146] 10.222.0.55 () {56 vars in 1374 bytes} [Fri Sep 23 15:00:51 2022] GET /identity/v3/role_assignments?user.id=8fe3d6d737b4427081967f517696d2be&scope.project.id=8b33b96fe121426d87db440adfed911c&role.id=8e2919e79df040e8926f21607818b58c => generated 259 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:00:54.139256 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cfaaf0-88e4-4321-a4e1-fd190b151d33 None None] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:54.139777 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cfaaf0-88e4-4321-a4e1-fd190b151d33 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:54.139777 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cfaaf0-88e4-4321-a4e1-fd190b151d33 None None] PATH_INFO: `/` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:54.141135 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 558/1147] 10.222.0.55 () {54 vars in 927 bytes} [Fri Sep 23 15:00:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:00:54.146354 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f0cbd2-71e4-4052-a704-564dcbb59b71 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:54.146742 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f0cbd2-71e4-4052-a704-564dcbb59b71 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:54.146913 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f0cbd2-71e4-4052-a704-564dcbb59b71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:54.166310 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-b3f0cbd2-71e4-4052-a704-564dcbb59b71 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:00:54.167494 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-b3f0cbd2-71e4-4052-a704-564dcbb59b71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:00:54.235317 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 590/1148] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Sep 23 15:00:54 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 23 15:00:54.509207 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cfaaf0-88e4-4321-a4e1-fd190b151d33 None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:00:54.554737 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:00:54.614645 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cfaaf0-88e4-4321-a4e1-fd190b151d33 None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:00:54.648992 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990e2cf8-d5c1-4287-82d1-26d5a9f07e27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:00:54.649608 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990e2cf8-d5c1-4287-82d1-26d5a9f07e27 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:54.649986 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990e2cf8-d5c1-4287-82d1-26d5a9f07e27 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:54.650766 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990e2cf8-d5c1-4287-82d1-26d5a9f07e27 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:54.660942 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990e2cf8-d5c1-4287-82d1-26d5a9f07e27 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 15:00:54.665691 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application [None req-990e2cf8-d5c1-4287-82d1-26d5a9f07e27 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 23 15:00:54.665691 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:00:54.665691 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:00:54.665691 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:00:54.665691 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:00:54.665691 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:00:54.665691 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:00:54.665691 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:00:54.665691 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:00:54.665691 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:00:54.665691 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:00:54.665691 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:00:54.665691 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 15:00:54.665691 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 15:00:54.665691 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 15:00:54.665691 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 15:00:54.667065 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 15:00:54.667065 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 15:00:54.667065 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 15:00:54.667065 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 15:00:54.667065 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:00:54.667065 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:00:54.667065 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:00:54.667065 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:00:54.667065 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:00:54.667065 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:00:54.667065 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:00:54.667065 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application with Lock( Sep 23 15:00:54.667065 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:00:54.667065 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:00:54.667065 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:00:54.667065 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:00:54.668037 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:00:54.668037 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:00:54.668037 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:00:54.668037 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:00:54.668037 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 15:00:54.668037 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 15:00:54.668037 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 15:00:54.668037 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 15:00:54.668037 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 15:00:54.668037 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 15:00:54.668037 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 23 15:00:54.668037 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Sep 23 15:00:54.668037 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 559/1149] 10.222.0.55 () {56 vars in 1096 bytes} [Fri Sep 23 15:00:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:00:54.673168 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f0cbd2-71e4-4052-a704-564dcbb59b71 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:00:54.749899 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:00:54.768572 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f0cbd2-71e4-4052-a704-564dcbb59b71 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:00:54.802723 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1ae46e-8b70-4a23-b30b-db4e56931c23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:00:54.803323 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1ae46e-8b70-4a23-b30b-db4e56931c23 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:54.803600 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1ae46e-8b70-4a23-b30b-db4e56931c23 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:54.803869 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1ae46e-8b70-4a23-b30b-db4e56931c23 None admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:54.804554 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1ae46e-8b70-4a23-b30b-db4e56931c23 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:00:54.804857 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1ae46e-8b70-4a23-b30b-db4e56931c23 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:00:54.806227 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1ae46e-8b70-4a23-b30b-db4e56931c23 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:00:54.814108 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 591/1150] 10.222.0.55 () {56 vars in 1099 bytes} [Fri Sep 23 15:00:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:00:54.820183 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990e2cf8-d5c1-4287-82d1-26d5a9f07e27 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:00:54.919907 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:00:54.922772 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990e2cf8-d5c1-4287-82d1-26d5a9f07e27 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:00:54.956950 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbcb526-5012-4d59-8010-3707b2f19244 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:00:54.957639 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbcb526-5012-4d59-8010-3707b2f19244 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:54.957868 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbcb526-5012-4d59-8010-3707b2f19244 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:54.958145 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbcb526-5012-4d59-8010-3707b2f19244 None admin] PATH_INFO: `/v3/users/8fe3d6d737b4427081967f517696d2be` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:54.959276 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbcb526-5012-4d59-8010-3707b2f19244 None admin] RBAC: Authorizing `identity:get_user(user_id=8fe3d6d737b4427081967f517696d2be)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:00:54.960869 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbcb526-5012-4d59-8010-3707b2f19244 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:00:54.962436 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 560/1151] 10.222.0.55 () {56 vars in 1177 bytes} [Fri Sep 23 15:00:54 2022] GET /identity/v3/users/8fe3d6d737b4427081967f517696d2be => generated 286 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:00:54.968974 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1ae46e-8b70-4a23-b30b-db4e56931c23 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:00:55.068897 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1ae46e-8b70-4a23-b30b-db4e56931c23 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:00:55.101922 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b210c325-c35b-4822-a925-32fc69149f3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:00:55.102583 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b210c325-c35b-4822-a925-32fc69149f3c None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:55.102868 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b210c325-c35b-4822-a925-32fc69149f3c None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:55.103332 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b210c325-c35b-4822-a925-32fc69149f3c None admin] PATH_INFO: `/v3/projects/8b33b96fe121426d87db440adfed911c` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:55.104688 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b210c325-c35b-4822-a925-32fc69149f3c None admin] RBAC: Authorizing `identity:get_project(project_id=8b33b96fe121426d87db440adfed911c)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:00:55.106137 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b210c325-c35b-4822-a925-32fc69149f3c None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:00:55.107762 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 592/1152] 10.222.0.55 () {56 vars in 1186 bytes} [Fri Sep 23 15:00:54 2022] GET /identity/v3/projects/8b33b96fe121426d87db440adfed911c => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:00:55.113795 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbcb526-5012-4d59-8010-3707b2f19244 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:00:55.217682 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbcb526-5012-4d59-8010-3707b2f19244 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:00:55.253173 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add10878-e43e-46a7-956f-2d2d82c3648e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:00:55.253832 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add10878-e43e-46a7-956f-2d2d82c3648e None admin] REQUEST_METHOD: `PUT` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:55.254113 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add10878-e43e-46a7-956f-2d2d82c3648e None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:55.254387 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add10878-e43e-46a7-956f-2d2d82c3648e None admin] PATH_INFO: `/v3/projects/8b33b96fe121426d87db440adfed911c/users/8fe3d6d737b4427081967f517696d2be/roles/8e2919e79df040e8926f21607818b58c` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:55.256303 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add10878-e43e-46a7-956f-2d2d82c3648e None admin] RBAC: Authorizing `identity:create_grant(project_id=8b33b96fe121426d87db440adfed911c, user_id=8fe3d6d737b4427081967f517696d2be, role_id=8e2919e79df040e8926f21607818b58c)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:00:55.257765 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add10878-e43e-46a7-956f-2d2d82c3648e None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:00:55.366971 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 561/1153] 10.222.0.55 () {58 vars in 1439 bytes} [Fri Sep 23 15:00:55 2022] PUT /identity/v3/projects/8b33b96fe121426d87db440adfed911c/users/8fe3d6d737b4427081967f517696d2be/roles/8e2919e79df040e8926f21607818b58c => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:00:57.538812 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a8604f-8767-4c2e-a69e-ba87f4cba8b8 None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:57.539378 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a8604f-8767-4c2e-a69e-ba87f4cba8b8 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:57.539378 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a8604f-8767-4c2e-a69e-ba87f4cba8b8 None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:57.541343 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 593/1154] 10.222.0.55 () {54 vars in 927 bytes} [Fri Sep 23 15:00:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:00:57.546666 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc708e3-6ee6-4baf-8922-52c116fde9f9 None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:57.547061 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc708e3-6ee6-4baf-8922-52c116fde9f9 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:57.547223 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc708e3-6ee6-4baf-8922-52c116fde9f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:57.565804 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-9bc708e3-6ee6-4baf-8922-52c116fde9f9 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:00:57.566985 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-9bc708e3-6ee6-4baf-8922-52c116fde9f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:00:57.589326 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:00:57.632141 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 562/1155] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Sep 23 15:00:57 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 23 15:00:57.903188 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a8604f-8767-4c2e-a69e-ba87f4cba8b8 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:00:57.916928 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:00:58.003771 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a8604f-8767-4c2e-a69e-ba87f4cba8b8 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:00:58.038677 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bea083-8974-4a54-bcfa-242e4cc77918 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:00:58.039301 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bea083-8974-4a54-bcfa-242e4cc77918 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:58.039931 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bea083-8974-4a54-bcfa-242e4cc77918 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:58.039931 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bea083-8974-4a54-bcfa-242e4cc77918 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:58.051501 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bea083-8974-4a54-bcfa-242e4cc77918 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 15:00:58.056783 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-e9bea083-8974-4a54-bcfa-242e4cc77918 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 23 15:00:58.056783 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:00:58.056783 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:00:58.056783 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:00:58.056783 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:00:58.056783 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:00:58.056783 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:00:58.056783 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:00:58.056783 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:00:58.056783 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:00:58.056783 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:00:58.056783 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:00:58.056783 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 15:00:58.056783 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 15:00:58.056783 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 15:00:58.056783 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 15:00:58.058010 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 15:00:58.058010 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 15:00:58.058010 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 15:00:58.058010 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 15:00:58.058010 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:00:58.058010 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:00:58.058010 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:00:58.058010 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:00:58.058010 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:00:58.058010 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:00:58.058010 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:00:58.058010 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Sep 23 15:00:58.058010 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:00:58.058010 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:00:58.058010 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:00:58.058010 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:00:58.059308 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:00:58.059308 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:00:58.059308 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:00:58.059308 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:00:58.059308 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 15:00:58.059308 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 15:00:58.059308 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 15:00:58.059308 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 15:00:58.059308 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 15:00:58.059308 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 15:00:58.059308 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 23 15:00:58.059308 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Sep 23 15:00:58.059308 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 594/1156] 10.222.0.55 () {56 vars in 1096 bytes} [Fri Sep 23 15:00:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:00:58.064117 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc708e3-6ee6-4baf-8922-52c116fde9f9 None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:00:58.166418 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc708e3-6ee6-4baf-8922-52c116fde9f9 None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:00:58.199111 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c19324-1c7d-4940-a353-8509e33d31b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:00:58.199708 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c19324-1c7d-4940-a353-8509e33d31b0 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:58.200008 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c19324-1c7d-4940-a353-8509e33d31b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:58.200304 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c19324-1c7d-4940-a353-8509e33d31b0 None admin] PATH_INFO: `/v3/roles` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:58.200952 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c19324-1c7d-4940-a353-8509e33d31b0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:00:58.201250 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c19324-1c7d-4940-a353-8509e33d31b0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:00:58.202597 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c19324-1c7d-4940-a353-8509e33d31b0 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:00:58.210312 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 563/1157] 10.222.0.55 () {56 vars in 1099 bytes} [Fri Sep 23 15:00:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:00:58.217167 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bea083-8974-4a54-bcfa-242e4cc77918 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:00:58.287922 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:00:58.318818 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bea083-8974-4a54-bcfa-242e4cc77918 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:00:58.354191 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d050a5dc-3750-4700-8a63-2e69df94cb3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:00:58.354870 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d050a5dc-3750-4700-8a63-2e69df94cb3e None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:58.355156 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d050a5dc-3750-4700-8a63-2e69df94cb3e None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:58.355464 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d050a5dc-3750-4700-8a63-2e69df94cb3e None admin] PATH_INFO: `/v3/users/8fe3d6d737b4427081967f517696d2be` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:58.356579 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d050a5dc-3750-4700-8a63-2e69df94cb3e None admin] RBAC: Authorizing `identity:get_user(user_id=8fe3d6d737b4427081967f517696d2be)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:00:58.358007 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d050a5dc-3750-4700-8a63-2e69df94cb3e None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:00:58.359682 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 595/1158] 10.222.0.55 () {56 vars in 1177 bytes} [Fri Sep 23 15:00:58 2022] GET /identity/v3/users/8fe3d6d737b4427081967f517696d2be => generated 286 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 15:00:58.365551 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c19324-1c7d-4940-a353-8509e33d31b0 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:00:58.388135 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:00:58.464463 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c19324-1c7d-4940-a353-8509e33d31b0 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:00:58.497099 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707ccd72-3f9d-4afa-bedd-d45552cb951e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:00:58.497687 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707ccd72-3f9d-4afa-bedd-d45552cb951e None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:58.497960 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707ccd72-3f9d-4afa-bedd-d45552cb951e None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:58.498226 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707ccd72-3f9d-4afa-bedd-d45552cb951e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:58.565792 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707ccd72-3f9d-4afa-bedd-d45552cb951e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:00:58.567232 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707ccd72-3f9d-4afa-bedd-d45552cb951e None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:00:58.589762 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:00:58.665604 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 564/1159] 10.222.0.55 () {58 vars in 1282 bytes} [Fri Sep 23 15:00:58 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 23 15:00:58.672262 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d050a5dc-3750-4700-8a63-2e69df94cb3e None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:00:58.771656 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d050a5dc-3750-4700-8a63-2e69df94cb3e None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:00:58.805062 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69054215-7dff-4d67-b9a9-a6c23dbcf83e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:00:58.805683 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69054215-7dff-4d67-b9a9-a6c23dbcf83e None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:58.805967 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69054215-7dff-4d67-b9a9-a6c23dbcf83e None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:58.806996 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69054215-7dff-4d67-b9a9-a6c23dbcf83e None admin] PATH_INFO: `/v3/projects/8b33b96fe121426d87db440adfed911c` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:58.808362 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69054215-7dff-4d67-b9a9-a6c23dbcf83e None admin] RBAC: Authorizing `identity:get_project(project_id=8b33b96fe121426d87db440adfed911c)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:00:58.809784 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69054215-7dff-4d67-b9a9-a6c23dbcf83e None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:00:58.811462 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 596/1160] 10.222.0.55 () {56 vars in 1186 bytes} [Fri Sep 23 15:00:58 2022] GET /identity/v3/projects/8b33b96fe121426d87db440adfed911c => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:00:58.817965 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707ccd72-3f9d-4afa-bedd-d45552cb951e None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:00:58.914847 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707ccd72-3f9d-4afa-bedd-d45552cb951e None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:00:58.934913 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:00:58.948837 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59350d4-3ce8-4a0f-adc6-98fc2af507f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:00:58.949436 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59350d4-3ce8-4a0f-adc6-98fc2af507f2 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:00:58.949710 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59350d4-3ce8-4a0f-adc6-98fc2af507f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:00:58.949980 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59350d4-3ce8-4a0f-adc6-98fc2af507f2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:00:58.950688 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59350d4-3ce8-4a0f-adc6-98fc2af507f2 None admin] RBAC: Adding query filter params (role.id=8e2919e79df040e8926f21607818b58c, scope.project.id=8b33b96fe121426d87db440adfed911c, user.id=8fe3d6d737b4427081967f517696d2be) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:00:58.950991 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59350d4-3ce8-4a0f-adc6-98fc2af507f2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:00:58.952426 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59350d4-3ce8-4a0f-adc6-98fc2af507f2 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:00:58.958058 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 565/1161] 10.222.0.55 () {56 vars in 1374 bytes} [Fri Sep 23 15:00:58 2022] GET /identity/v3/role_assignments?user.id=8fe3d6d737b4427081967f517696d2be&scope.project.id=8b33b96fe121426d87db440adfed911c&role.id=8e2919e79df040e8926f21607818b58c => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:01:01.128614 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba03c42f-5de9-4d2a-92d4-97356e4b2440 None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:01:01.129130 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba03c42f-5de9-4d2a-92d4-97356e4b2440 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:01:01.129249 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba03c42f-5de9-4d2a-92d4-97356e4b2440 None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:01:01.130487 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 597/1162] 10.222.0.55 () {54 vars in 927 bytes} [Fri Sep 23 15:01:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:01:01.135770 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64296e7-8014-4a82-b4d3-5af2c1fee4d0 None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:01:01.136196 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64296e7-8014-4a82-b4d3-5af2c1fee4d0 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:01:01.136357 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64296e7-8014-4a82-b4d3-5af2c1fee4d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:01:01.155019 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-d64296e7-8014-4a82-b4d3-5af2c1fee4d0 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:01:01.156237 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-d64296e7-8014-4a82-b4d3-5af2c1fee4d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:01:01.222131 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 566/1163] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Sep 23 15:01:01 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 23 15:01:01.488642 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba03c42f-5de9-4d2a-92d4-97356e4b2440 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:01:01.498061 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:01:01.588075 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba03c42f-5de9-4d2a-92d4-97356e4b2440 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:01:01.620080 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d110f0-688c-41c0-aab3-9992d5eaaf1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:01:01.620677 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d110f0-688c-41c0-aab3-9992d5eaaf1b None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:01:01.620955 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d110f0-688c-41c0-aab3-9992d5eaaf1b None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:01:01.621223 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d110f0-688c-41c0-aab3-9992d5eaaf1b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:01:01.686626 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d110f0-688c-41c0-aab3-9992d5eaaf1b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:01:01.688076 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d110f0-688c-41c0-aab3-9992d5eaaf1b None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:01:01.695397 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:01:01.783110 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 598/1164] 10.222.0.55 () {58 vars in 1282 bytes} [Fri Sep 23 15:01:01 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 23 15:01:01.789402 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64296e7-8014-4a82-b4d3-5af2c1fee4d0 None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:01:01.889260 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64296e7-8014-4a82-b4d3-5af2c1fee4d0 None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:01:01.903271 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:01:01.923409 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a09d93-5b74-4f97-9ba3-511abe6d5df6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:01:01.924064 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a09d93-5b74-4f97-9ba3-511abe6d5df6 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:01:01.924348 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a09d93-5b74-4f97-9ba3-511abe6d5df6 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:01:01.924621 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a09d93-5b74-4f97-9ba3-511abe6d5df6 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:01:01.931854 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a09d93-5b74-4f97-9ba3-511abe6d5df6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:01:01.933673 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a09d93-5b74-4f97-9ba3-511abe6d5df6 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:01:01.945586 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application [None req-d9a09d93-5b74-4f97-9ba3-511abe6d5df6 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 23 15:01:01.945586 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:01:01.945586 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 23 15:01:01.945586 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 23 15:01:01.945586 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 23 15:01:01.945586 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 23 15:01:01.945586 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 23 15:01:01.945586 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 23 15:01:01.945586 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Sep 23 15:01:01.945586 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Sep 23 15:01:01.945586 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 23 15:01:01.945586 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Sep 23 15:01:01.945586 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:01:01.945586 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:01:01.945586 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:01:01.945586 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:01:01.945586 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:01:01.947616 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:01:01.947616 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:01:01.947616 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:01:01.947616 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:01:01.947616 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:01:01.947616 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:01:01.947616 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 15:01:01.947616 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 15:01:01.947616 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 15:01:01.947616 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 15:01:01.947616 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:01:01.947616 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:01:01.947616 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:01:01.947616 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:01:01.947616 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:01:01.947616 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:01:01.948829 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:01:01.948829 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application with Lock( Sep 23 15:01:01.948829 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:01:01.948829 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:01:01.948829 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:01:01.948829 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:01:01.948829 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:01:01.948829 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:01:01.948829 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:01:01.948829 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:01:01.948829 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 23 15:01:01.948829 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 15:01:01.948829 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 23 15:01:01.948829 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Sep 23 15:01:01.948829 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 567/1165] 10.222.0.55 () {56 vars in 1117 bytes} [Fri Sep 23 15:01:01 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:01:01.953534 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d110f0-688c-41c0-aab3-9992d5eaaf1b None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:01:02.049151 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d110f0-688c-41c0-aab3-9992d5eaaf1b None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:01:02.056095 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:01:02.083338 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2c5c34-964e-4d37-a914-0cd781ab83f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:01:02.083943 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2c5c34-964e-4d37-a914-0cd781ab83f1 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:01:02.084249 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2c5c34-964e-4d37-a914-0cd781ab83f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:01:02.084519 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2c5c34-964e-4d37-a914-0cd781ab83f1 None admin] PATH_INFO: `/v3/domains` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:01:02.085150 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2c5c34-964e-4d37-a914-0cd781ab83f1 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:01:02.085498 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2c5c34-964e-4d37-a914-0cd781ab83f1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:01:02.086792 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2c5c34-964e-4d37-a914-0cd781ab83f1 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:01:02.093836 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 599/1166] 10.222.0.55 () {56 vars in 1115 bytes} [Fri Sep 23 15:01:01 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:01:04.268769 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aae18a5-74db-4ecd-bd80-21988d4eeb1f None None] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:01:04.269317 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aae18a5-74db-4ecd-bd80-21988d4eeb1f None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:01:04.269317 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aae18a5-74db-4ecd-bd80-21988d4eeb1f None None] PATH_INFO: `/` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:01:04.270643 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 568/1167] 10.222.0.55 () {54 vars in 927 bytes} [Fri Sep 23 15:01:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:01:04.276626 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba57a53-3112-4714-9257-2091e7296b5e None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:01:04.277054 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba57a53-3112-4714-9257-2091e7296b5e None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:01:04.277156 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba57a53-3112-4714-9257-2091e7296b5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:01:04.295547 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-1ba57a53-3112-4714-9257-2091e7296b5e None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:01:04.296767 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-1ba57a53-3112-4714-9257-2091e7296b5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:01:04.369081 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 600/1168] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Sep 23 15:01:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 23 15:01:04.669513 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aae18a5-74db-4ecd-bd80-21988d4eeb1f None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:01:04.780690 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aae18a5-74db-4ecd-bd80-21988d4eeb1f None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:01:04.815130 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa56dcef-8d76-4325-b33a-5affe0ae189d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:01:04.815744 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa56dcef-8d76-4325-b33a-5affe0ae189d None admin] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:01:04.816049 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa56dcef-8d76-4325-b33a-5affe0ae189d None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:01:04.816325 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa56dcef-8d76-4325-b33a-5affe0ae189d None admin] PATH_INFO: `/v3/domains` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:01:04.816990 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa56dcef-8d76-4325-b33a-5affe0ae189d None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:01:04.818389 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa56dcef-8d76-4325-b33a-5affe0ae189d None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:01:04.991746 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 569/1169] 10.222.0.55 () {60 vars in 1138 bytes} [Fri Sep 23 15:01:04 2022] POST /identity/v3/domains => generated 267 bytes in 323 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:01:07.200392 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af59ad9d-f6ba-45cf-8e6b-50cf06c4d73a None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:01:07.200923 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af59ad9d-f6ba-45cf-8e6b-50cf06c4d73a None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:01:07.200923 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af59ad9d-f6ba-45cf-8e6b-50cf06c4d73a None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:01:07.202260 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 601/1170] 10.222.0.55 () {54 vars in 927 bytes} [Fri Sep 23 15:01:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:01:07.207417 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb51d5a-743c-4ce4-8e56-2088f5924738 None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:01:07.207839 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb51d5a-743c-4ce4-8e56-2088f5924738 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:01:07.207989 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb51d5a-743c-4ce4-8e56-2088f5924738 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:01:07.226726 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-ffb51d5a-743c-4ce4-8e56-2088f5924738 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:01:07.227953 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-ffb51d5a-743c-4ce4-8e56-2088f5924738 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:01:07.256587 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:01:07.298927 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 570/1171] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Sep 23 15:01:07 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 23 15:01:07.569756 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af59ad9d-f6ba-45cf-8e6b-50cf06c4d73a None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:01:07.669806 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:01:07.673709 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af59ad9d-f6ba-45cf-8e6b-50cf06c4d73a None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:01:07.707594 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8dae59-b984-4c8f-997f-2c81ee75ff91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:01:07.708263 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8dae59-b984-4c8f-997f-2c81ee75ff91 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:01:07.708551 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8dae59-b984-4c8f-997f-2c81ee75ff91 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:01:07.708826 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8dae59-b984-4c8f-997f-2c81ee75ff91 None admin] PATH_INFO: `/v3/domains/131c60f1b6644bfcb4475a6844e01e17` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:01:07.720000 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8dae59-b984-4c8f-997f-2c81ee75ff91 None admin] RBAC: Authorizing `identity:get_domain(domain_id=131c60f1b6644bfcb4475a6844e01e17)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:01:07.721462 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8dae59-b984-4c8f-997f-2c81ee75ff91 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:01:07.723106 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 602/1172] 10.222.0.55 () {56 vars in 1183 bytes} [Fri Sep 23 15:01:07 2022] GET /identity/v3/domains/131c60f1b6644bfcb4475a6844e01e17 => generated 267 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:01:07.729808 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb51d5a-743c-4ce4-8e56-2088f5924738 None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:01:07.828080 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb51d5a-743c-4ce4-8e56-2088f5924738 None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:01:07.861793 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd80306-1ead-4db1-bdaa-666d7bf8bfcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:01:07.862400 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd80306-1ead-4db1-bdaa-666d7bf8bfcd None admin] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:01:07.862678 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd80306-1ead-4db1-bdaa-666d7bf8bfcd None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:01:07.862949 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd80306-1ead-4db1-bdaa-666d7bf8bfcd None admin] PATH_INFO: `/v3/projects` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:01:07.863597 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd80306-1ead-4db1-bdaa-666d7bf8bfcd None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:01:07.865057 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd80306-1ead-4db1-bdaa-666d7bf8bfcd None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:01:07.996873 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 571/1173] 10.222.0.55 () {60 vars in 1141 bytes} [Fri Sep 23 15:01:07 2022] POST /identity/v3/projects => generated 361 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:01:10.181993 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf82721-6828-4acb-a1d4-b3eed3e9c401 None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:01:10.182515 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf82721-6828-4acb-a1d4-b3eed3e9c401 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:01:10.182606 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf82721-6828-4acb-a1d4-b3eed3e9c401 None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:01:10.183900 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 603/1174] 10.222.0.55 () {54 vars in 927 bytes} [Fri Sep 23 15:01:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:01:10.189224 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23deca9b-54d7-415a-aea2-380b624dda9b None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:01:10.189668 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23deca9b-54d7-415a-aea2-380b624dda9b None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:01:10.189766 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23deca9b-54d7-415a-aea2-380b624dda9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:01:10.208252 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-23deca9b-54d7-415a-aea2-380b624dda9b None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:01:10.209414 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-23deca9b-54d7-415a-aea2-380b624dda9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:01:10.219815 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:01:10.274335 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 572/1175] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Sep 23 15:01:10 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 23 15:01:10.546251 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf82721-6828-4acb-a1d4-b3eed3e9c401 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:01:10.645821 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf82721-6828-4acb-a1d4-b3eed3e9c401 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:01:10.679108 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567b65f7-f51c-4871-aabb-288f31a12c8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:01:10.679734 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567b65f7-f51c-4871-aabb-288f31a12c8d None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:01:10.680041 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567b65f7-f51c-4871-aabb-288f31a12c8d None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:01:10.680320 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567b65f7-f51c-4871-aabb-288f31a12c8d None admin] PATH_INFO: `/v3/domains/131c60f1b6644bfcb4475a6844e01e17` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:01:10.681611 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567b65f7-f51c-4871-aabb-288f31a12c8d None admin] RBAC: Authorizing `identity:get_domain(domain_id=131c60f1b6644bfcb4475a6844e01e17)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:01:10.683031 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567b65f7-f51c-4871-aabb-288f31a12c8d None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:01:10.684611 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 604/1176] 10.222.0.55 () {56 vars in 1183 bytes} [Fri Sep 23 15:01:10 2022] GET /identity/v3/domains/131c60f1b6644bfcb4475a6844e01e17 => generated 267 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:01:10.690557 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23deca9b-54d7-415a-aea2-380b624dda9b None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:01:10.788467 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23deca9b-54d7-415a-aea2-380b624dda9b None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:01:10.821709 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d54c95b-bc3b-4fb5-b521-c79d87f9f622 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:01:10.822378 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d54c95b-bc3b-4fb5-b521-c79d87f9f622 None admin] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:01:10.822591 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d54c95b-bc3b-4fb5-b521-c79d87f9f622 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:01:10.822840 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d54c95b-bc3b-4fb5-b521-c79d87f9f622 None admin] PATH_INFO: `/v3/users` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:01:10.823477 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d54c95b-bc3b-4fb5-b521-c79d87f9f622 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:01:10.824944 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d54c95b-bc3b-4fb5-b521-c79d87f9f622 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:01:10.972725 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 573/1177] 10.222.0.55 () {60 vars in 1132 bytes} [Fri Sep 23 15:01:10 2022] POST /identity/v3/users => generated 311 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:01:13.177953 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3473982-bfe9-4948-b4a3-a8c077c33281 None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:01:13.178473 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3473982-bfe9-4948-b4a3-a8c077c33281 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:01:13.178600 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3473982-bfe9-4948-b4a3-a8c077c33281 None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:01:13.179850 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 605/1178] 10.222.0.55 () {54 vars in 927 bytes} [Fri Sep 23 15:01:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:01:13.185772 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ea7aa6-afa7-415e-a051-8b983df4b4c3 None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:01:13.186189 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ea7aa6-afa7-415e-a051-8b983df4b4c3 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:01:13.186316 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ea7aa6-afa7-415e-a051-8b983df4b4c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:01:13.192573 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:01:13.205152 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-08ea7aa6-afa7-415e-a051-8b983df4b4c3 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:01:13.206371 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-08ea7aa6-afa7-415e-a051-8b983df4b4c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:01:13.270600 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 574/1179] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Sep 23 15:01:13 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 23 15:01:13.543816 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3473982-bfe9-4948-b4a3-a8c077c33281 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:01:13.573896 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:01:13.649387 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3473982-bfe9-4948-b4a3-a8c077c33281 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:01:13.684658 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2d322f-b489-4724-9071-5cc61d36e613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:01:13.685284 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2d322f-b489-4724-9071-5cc61d36e613 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:01:13.685568 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2d322f-b489-4724-9071-5cc61d36e613 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:01:13.685840 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2d322f-b489-4724-9071-5cc61d36e613 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:01:13.696815 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2d322f-b489-4724-9071-5cc61d36e613 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 15:01:13.702399 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-4d2d322f-b489-4724-9071-5cc61d36e613 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 23 15:01:13.702399 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:01:13.702399 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:01:13.702399 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:01:13.702399 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:01:13.702399 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:01:13.702399 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:01:13.702399 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:01:13.702399 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:01:13.702399 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:01:13.702399 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:01:13.702399 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:01:13.702399 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 15:01:13.702399 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 15:01:13.702399 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 15:01:13.702399 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 15:01:13.703498 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 15:01:13.703498 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 15:01:13.703498 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 15:01:13.703498 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 15:01:13.703498 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:01:13.703498 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:01:13.703498 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:01:13.703498 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:01:13.703498 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:01:13.703498 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:01:13.703498 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:01:13.703498 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Sep 23 15:01:13.703498 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:01:13.703498 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:01:13.703498 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:01:13.703498 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:01:13.704839 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:01:13.704839 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:01:13.704839 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:01:13.704839 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:01:13.704839 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 15:01:13.704839 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 15:01:13.704839 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 15:01:13.704839 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 15:01:13.704839 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 15:01:13.704839 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 15:01:13.704839 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 23 15:01:13.704839 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Sep 23 15:01:13.704839 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 606/1180] 10.222.0.55 () {56 vars in 1096 bytes} [Fri Sep 23 15:01:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:01:13.709589 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ea7aa6-afa7-415e-a051-8b983df4b4c3 None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:01:13.807232 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ea7aa6-afa7-415e-a051-8b983df4b4c3 None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:01:13.835104 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:01:13.840544 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6f37ed-f520-44aa-8ff6-1556610baa70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:01:13.841152 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6f37ed-f520-44aa-8ff6-1556610baa70 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:01:13.841429 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6f37ed-f520-44aa-8ff6-1556610baa70 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:01:13.841699 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6f37ed-f520-44aa-8ff6-1556610baa70 None admin] PATH_INFO: `/v3/roles` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:01:13.842351 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6f37ed-f520-44aa-8ff6-1556610baa70 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:01:13.842650 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6f37ed-f520-44aa-8ff6-1556610baa70 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:01:13.844025 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6f37ed-f520-44aa-8ff6-1556610baa70 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:01:13.851960 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 575/1181] 10.222.0.55 () {56 vars in 1099 bytes} [Fri Sep 23 15:01:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:01:13.858639 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2d322f-b489-4724-9071-5cc61d36e613 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:01:13.949377 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:01:13.961074 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2d322f-b489-4724-9071-5cc61d36e613 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:01:13.995624 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921fab96-162a-460d-82e9-b69499c4b92c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:01:13.996284 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921fab96-162a-460d-82e9-b69499c4b92c None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:01:13.996568 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921fab96-162a-460d-82e9-b69499c4b92c None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:01:13.996841 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921fab96-162a-460d-82e9-b69499c4b92c None admin] PATH_INFO: `/v3/users/f49a2da9db5243c6941ab6bfeddd8a1d` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:01:14.010161 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921fab96-162a-460d-82e9-b69499c4b92c None admin] RBAC: Authorizing `identity:get_user(user_id=f49a2da9db5243c6941ab6bfeddd8a1d)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:01:14.011577 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921fab96-162a-460d-82e9-b69499c4b92c None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:01:14.013177 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 607/1182] 10.222.0.55 () {56 vars in 1177 bytes} [Fri Sep 23 15:01:13 2022] GET /identity/v3/users/f49a2da9db5243c6941ab6bfeddd8a1d => generated 311 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:01:14.019027 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6f37ed-f520-44aa-8ff6-1556610baa70 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:01:14.118609 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6f37ed-f520-44aa-8ff6-1556610baa70 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:01:14.153196 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3ca533-f6be-4889-b9a4-528124110b6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:01:14.153807 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3ca533-f6be-4889-b9a4-528124110b6a None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:01:14.154088 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3ca533-f6be-4889-b9a4-528124110b6a None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:01:14.154363 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3ca533-f6be-4889-b9a4-528124110b6a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:01:14.213842 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:01:14.221759 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3ca533-f6be-4889-b9a4-528124110b6a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:01:14.227463 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3ca533-f6be-4889-b9a4-528124110b6a None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:01:14.322716 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 576/1183] 10.222.0.55 () {58 vars in 1282 bytes} [Fri Sep 23 15:01:14 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 23 15:01:14.329171 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921fab96-162a-460d-82e9-b69499c4b92c None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:01:14.429003 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921fab96-162a-460d-82e9-b69499c4b92c None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:01:14.463855 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60e99a1-962c-409d-92a1-193bf12a3e5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:01:14.464499 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60e99a1-962c-409d-92a1-193bf12a3e5e None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:01:14.464785 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60e99a1-962c-409d-92a1-193bf12a3e5e None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:01:14.465081 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60e99a1-962c-409d-92a1-193bf12a3e5e None admin] PATH_INFO: `/v3/projects/3808a78915a44e0594da8222173d3ea4` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:01:14.466192 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60e99a1-962c-409d-92a1-193bf12a3e5e None admin] RBAC: Authorizing `identity:get_project(project_id=3808a78915a44e0594da8222173d3ea4)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:01:14.467607 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60e99a1-962c-409d-92a1-193bf12a3e5e None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:01:14.469283 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 608/1184] 10.222.0.55 () {56 vars in 1186 bytes} [Fri Sep 23 15:01:14 2022] GET /identity/v3/projects/3808a78915a44e0594da8222173d3ea4 => generated 361 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:01:14.475443 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3ca533-f6be-4889-b9a4-528124110b6a None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:01:14.569257 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:01:14.576440 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3ca533-f6be-4889-b9a4-528124110b6a None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:01:14.609358 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3a613e-1e0e-4ea8-a2e5-05bfbf1536d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:01:14.609963 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3a613e-1e0e-4ea8-a2e5-05bfbf1536d3 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:01:14.610246 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3a613e-1e0e-4ea8-a2e5-05bfbf1536d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:01:14.610521 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3a613e-1e0e-4ea8-a2e5-05bfbf1536d3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:01:14.611279 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3a613e-1e0e-4ea8-a2e5-05bfbf1536d3 None admin] RBAC: Adding query filter params (role.id=8e2919e79df040e8926f21607818b58c, scope.project.id=3808a78915a44e0594da8222173d3ea4, user.id=f49a2da9db5243c6941ab6bfeddd8a1d) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:01:14.611791 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3a613e-1e0e-4ea8-a2e5-05bfbf1536d3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:01:14.613692 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3a613e-1e0e-4ea8-a2e5-05bfbf1536d3 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:01:14.619961 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 577/1185] 10.222.0.55 () {56 vars in 1374 bytes} [Fri Sep 23 15:01:14 2022] GET /identity/v3/role_assignments?user.id=f49a2da9db5243c6941ab6bfeddd8a1d&scope.project.id=3808a78915a44e0594da8222173d3ea4&role.id=8e2919e79df040e8926f21607818b58c => generated 259 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:01:16.788296 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e179316-0fb3-43c8-af82-9d172c2a05df None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:01:16.788832 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e179316-0fb3-43c8-af82-9d172c2a05df None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:01:16.788832 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e179316-0fb3-43c8-af82-9d172c2a05df None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:01:16.790179 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 609/1186] 10.222.0.55 () {54 vars in 927 bytes} [Fri Sep 23 15:01:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:01:16.795557 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a24bfb6-472c-44d9-8496-1ed3ebb312eb None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:01:16.795943 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a24bfb6-472c-44d9-8496-1ed3ebb312eb None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:01:16.796132 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a24bfb6-472c-44d9-8496-1ed3ebb312eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:01:16.814678 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-4a24bfb6-472c-44d9-8496-1ed3ebb312eb None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:01:16.815873 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-4a24bfb6-472c-44d9-8496-1ed3ebb312eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:01:16.881860 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 578/1187] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Sep 23 15:01:16 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 23 15:01:17.154578 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e179316-0fb3-43c8-af82-9d172c2a05df None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:01:17.170956 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:01:17.256045 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e179316-0fb3-43c8-af82-9d172c2a05df None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:01:17.289396 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b057b09-9009-4f8a-9874-d0d15e58a73d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:01:17.290023 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b057b09-9009-4f8a-9874-d0d15e58a73d None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:01:17.290303 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b057b09-9009-4f8a-9874-d0d15e58a73d None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:01:17.290577 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b057b09-9009-4f8a-9874-d0d15e58a73d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:01:17.300906 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b057b09-9009-4f8a-9874-d0d15e58a73d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 15:01:17.305623 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-0b057b09-9009-4f8a-9874-d0d15e58a73d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 23 15:01:17.305623 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:01:17.305623 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:01:17.305623 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:01:17.305623 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:01:17.305623 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:01:17.305623 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:01:17.305623 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:01:17.305623 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:01:17.305623 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:01:17.305623 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:01:17.305623 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:01:17.305623 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 15:01:17.305623 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 15:01:17.305623 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 15:01:17.305623 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 15:01:17.307218 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 15:01:17.307218 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 15:01:17.307218 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 15:01:17.307218 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 15:01:17.307218 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:01:17.307218 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:01:17.307218 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:01:17.307218 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:01:17.307218 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:01:17.307218 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:01:17.307218 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:01:17.307218 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Sep 23 15:01:17.307218 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:01:17.307218 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:01:17.307218 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:01:17.307218 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:01:17.308393 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:01:17.308393 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:01:17.308393 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:01:17.308393 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:01:17.308393 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 15:01:17.308393 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 15:01:17.308393 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 15:01:17.308393 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 15:01:17.308393 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 15:01:17.308393 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 15:01:17.308393 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 23 15:01:17.308393 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Sep 23 15:01:17.308393 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 610/1188] 10.222.0.55 () {56 vars in 1096 bytes} [Fri Sep 23 15:01:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:01:17.313324 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a24bfb6-472c-44d9-8496-1ed3ebb312eb None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:01:17.397793 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:01:17.410142 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a24bfb6-472c-44d9-8496-1ed3ebb312eb None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:01:17.444142 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1cbc18-52dd-453e-a095-3e5cfc0bb59b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:01:17.444750 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1cbc18-52dd-453e-a095-3e5cfc0bb59b None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:01:17.445047 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1cbc18-52dd-453e-a095-3e5cfc0bb59b None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:01:17.445304 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1cbc18-52dd-453e-a095-3e5cfc0bb59b None admin] PATH_INFO: `/v3/roles` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:01:17.445962 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1cbc18-52dd-453e-a095-3e5cfc0bb59b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:01:17.446264 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1cbc18-52dd-453e-a095-3e5cfc0bb59b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:01:17.447812 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1cbc18-52dd-453e-a095-3e5cfc0bb59b None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:01:17.455774 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 579/1189] 10.222.0.55 () {56 vars in 1099 bytes} [Fri Sep 23 15:01:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:01:17.461732 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b057b09-9009-4f8a-9874-d0d15e58a73d None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:01:17.537981 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:01:17.563054 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b057b09-9009-4f8a-9874-d0d15e58a73d None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:01:17.597593 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75fc4f8-585b-4608-9a82-7ee7a887bcbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:01:17.598216 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75fc4f8-585b-4608-9a82-7ee7a887bcbe None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:01:17.598498 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75fc4f8-585b-4608-9a82-7ee7a887bcbe None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:01:17.598771 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75fc4f8-585b-4608-9a82-7ee7a887bcbe None admin] PATH_INFO: `/v3/users/f49a2da9db5243c6941ab6bfeddd8a1d` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:01:17.599951 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75fc4f8-585b-4608-9a82-7ee7a887bcbe None admin] RBAC: Authorizing `identity:get_user(user_id=f49a2da9db5243c6941ab6bfeddd8a1d)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:01:17.601393 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75fc4f8-585b-4608-9a82-7ee7a887bcbe None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:01:17.602975 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 611/1190] 10.222.0.55 () {56 vars in 1177 bytes} [Fri Sep 23 15:01:17 2022] GET /identity/v3/users/f49a2da9db5243c6941ab6bfeddd8a1d => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:01:17.609506 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1cbc18-52dd-453e-a095-3e5cfc0bb59b None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:01:17.706970 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1cbc18-52dd-453e-a095-3e5cfc0bb59b None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:01:17.741077 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f148f847-b38a-43bf-a342-f514b12de865 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:01:17.741718 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f148f847-b38a-43bf-a342-f514b12de865 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:01:17.741997 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f148f847-b38a-43bf-a342-f514b12de865 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:01:17.742268 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f148f847-b38a-43bf-a342-f514b12de865 None admin] PATH_INFO: `/v3/projects/3808a78915a44e0594da8222173d3ea4` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:01:17.743368 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f148f847-b38a-43bf-a342-f514b12de865 None admin] RBAC: Authorizing `identity:get_project(project_id=3808a78915a44e0594da8222173d3ea4)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:01:17.744876 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f148f847-b38a-43bf-a342-f514b12de865 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:01:17.746483 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 580/1191] 10.222.0.55 () {56 vars in 1186 bytes} [Fri Sep 23 15:01:17 2022] GET /identity/v3/projects/3808a78915a44e0594da8222173d3ea4 => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:01:17.752541 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75fc4f8-585b-4608-9a82-7ee7a887bcbe None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:01:17.855874 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75fc4f8-585b-4608-9a82-7ee7a887bcbe None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:01:17.889556 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a8f7ca-8b17-427d-82fb-fcb1a65f2c9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:01:17.890204 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a8f7ca-8b17-427d-82fb-fcb1a65f2c9b None admin] REQUEST_METHOD: `PUT` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:01:17.890485 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a8f7ca-8b17-427d-82fb-fcb1a65f2c9b None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:01:17.890766 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a8f7ca-8b17-427d-82fb-fcb1a65f2c9b None admin] PATH_INFO: `/v3/projects/3808a78915a44e0594da8222173d3ea4/users/f49a2da9db5243c6941ab6bfeddd8a1d/roles/8e2919e79df040e8926f21607818b58c` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:01:17.892640 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a8f7ca-8b17-427d-82fb-fcb1a65f2c9b None admin] RBAC: Authorizing `identity:create_grant(project_id=3808a78915a44e0594da8222173d3ea4, user_id=f49a2da9db5243c6941ab6bfeddd8a1d, role_id=8e2919e79df040e8926f21607818b58c)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:01:17.894064 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a8f7ca-8b17-427d-82fb-fcb1a65f2c9b None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:01:17.995452 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 612/1192] 10.222.0.55 () {58 vars in 1439 bytes} [Fri Sep 23 15:01:17 2022] PUT /identity/v3/projects/3808a78915a44e0594da8222173d3ea4/users/f49a2da9db5243c6941ab6bfeddd8a1d/roles/8e2919e79df040e8926f21607818b58c => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:01:20.123548 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b684b8b-c6ca-4bd5-9571-7840f7a4c11d None None] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:01:20.124034 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b684b8b-c6ca-4bd5-9571-7840f7a4c11d None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:01:20.124151 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b684b8b-c6ca-4bd5-9571-7840f7a4c11d None None] PATH_INFO: `/` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:01:20.125457 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 581/1193] 10.222.0.55 () {54 vars in 927 bytes} [Fri Sep 23 15:01:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:01:20.130578 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ee10e9-e6af-4116-b24d-4d814db1920f None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:01:20.130957 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ee10e9-e6af-4116-b24d-4d814db1920f None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:01:20.131115 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ee10e9-e6af-4116-b24d-4d814db1920f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:01:20.137491 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:01:20.149290 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-72ee10e9-e6af-4116-b24d-4d814db1920f None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:01:20.152443 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-72ee10e9-e6af-4116-b24d-4d814db1920f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:01:20.216046 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 613/1194] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Sep 23 15:01:20 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 23 15:01:20.485843 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b684b8b-c6ca-4bd5-9571-7840f7a4c11d None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:01:20.507947 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:01:20.594481 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b684b8b-c6ca-4bd5-9571-7840f7a4c11d None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:01:20.628144 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340f45c4-e1ca-4933-bcda-4336b1e0911a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:01:20.628775 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340f45c4-e1ca-4933-bcda-4336b1e0911a None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:01:20.629058 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340f45c4-e1ca-4933-bcda-4336b1e0911a None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:01:20.629336 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340f45c4-e1ca-4933-bcda-4336b1e0911a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:01:20.640974 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340f45c4-e1ca-4933-bcda-4336b1e0911a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 15:01:20.645764 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application [None req-340f45c4-e1ca-4933-bcda-4336b1e0911a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 23 15:01:20.645764 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:01:20.645764 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:01:20.645764 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:01:20.645764 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:01:20.645764 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:01:20.645764 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:01:20.645764 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:01:20.645764 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:01:20.645764 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:01:20.645764 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:01:20.645764 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:01:20.645764 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 15:01:20.645764 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 15:01:20.645764 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 15:01:20.645764 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 15:01:20.646930 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 15:01:20.646930 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 15:01:20.646930 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 15:01:20.646930 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 15:01:20.646930 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:01:20.646930 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:01:20.646930 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:01:20.646930 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:01:20.646930 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:01:20.646930 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:01:20.646930 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:01:20.646930 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application with Lock( Sep 23 15:01:20.646930 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:01:20.646930 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:01:20.646930 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:01:20.646930 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:01:20.647847 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:01:20.647847 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:01:20.647847 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:01:20.647847 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:01:20.647847 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 15:01:20.647847 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 15:01:20.647847 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 15:01:20.647847 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 15:01:20.647847 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 15:01:20.647847 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 15:01:20.647847 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 23 15:01:20.647847 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Sep 23 15:01:20.647847 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 582/1195] 10.222.0.55 () {56 vars in 1096 bytes} [Fri Sep 23 15:01:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:01:20.653253 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ee10e9-e6af-4116-b24d-4d814db1920f None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:01:20.749570 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ee10e9-e6af-4116-b24d-4d814db1920f None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:01:20.778770 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:01:20.784438 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f422a346-223f-4e5d-abfa-a72b827f5f07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:01:20.789381 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f422a346-223f-4e5d-abfa-a72b827f5f07 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:01:20.789381 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f422a346-223f-4e5d-abfa-a72b827f5f07 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:01:20.789381 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f422a346-223f-4e5d-abfa-a72b827f5f07 None admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:01:20.789381 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f422a346-223f-4e5d-abfa-a72b827f5f07 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:01:20.789381 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f422a346-223f-4e5d-abfa-a72b827f5f07 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:01:20.789381 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f422a346-223f-4e5d-abfa-a72b827f5f07 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:01:20.796588 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 614/1196] 10.222.0.55 () {56 vars in 1099 bytes} [Fri Sep 23 15:01:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:01:20.802725 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340f45c4-e1ca-4933-bcda-4336b1e0911a None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:01:20.881734 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:01:20.902731 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340f45c4-e1ca-4933-bcda-4336b1e0911a None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:01:20.936677 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316727b1-04da-4584-bed8-8b11d40ccc09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:01:20.937298 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316727b1-04da-4584-bed8-8b11d40ccc09 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:01:20.937578 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316727b1-04da-4584-bed8-8b11d40ccc09 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:01:20.937851 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316727b1-04da-4584-bed8-8b11d40ccc09 None admin] PATH_INFO: `/v3/users/f49a2da9db5243c6941ab6bfeddd8a1d` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:01:20.938961 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316727b1-04da-4584-bed8-8b11d40ccc09 None admin] RBAC: Authorizing `identity:get_user(user_id=f49a2da9db5243c6941ab6bfeddd8a1d)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:01:20.940463 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316727b1-04da-4584-bed8-8b11d40ccc09 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:01:20.943899 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 583/1197] 10.222.0.55 () {56 vars in 1177 bytes} [Fri Sep 23 15:01:20 2022] GET /identity/v3/users/f49a2da9db5243c6941ab6bfeddd8a1d => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:01:20.948049 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f422a346-223f-4e5d-abfa-a72b827f5f07 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:01:21.045780 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f422a346-223f-4e5d-abfa-a72b827f5f07 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:01:21.079222 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca53fa96-baa3-4cf4-92ca-d5b5e4d0b47e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:01:21.079832 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca53fa96-baa3-4cf4-92ca-d5b5e4d0b47e None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:01:21.080511 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca53fa96-baa3-4cf4-92ca-d5b5e4d0b47e None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:01:21.080511 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca53fa96-baa3-4cf4-92ca-d5b5e4d0b47e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:01:21.139451 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:01:21.146589 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca53fa96-baa3-4cf4-92ca-d5b5e4d0b47e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:01:21.148103 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca53fa96-baa3-4cf4-92ca-d5b5e4d0b47e None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:01:21.244168 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 615/1198] 10.222.0.55 () {58 vars in 1282 bytes} [Fri Sep 23 15:01:20 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 23 15:01:21.250782 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316727b1-04da-4584-bed8-8b11d40ccc09 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:01:21.350206 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316727b1-04da-4584-bed8-8b11d40ccc09 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:01:21.383145 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa55e506-227c-46e4-87ae-28bf045cb048 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:01:21.383766 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa55e506-227c-46e4-87ae-28bf045cb048 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:01:21.384069 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa55e506-227c-46e4-87ae-28bf045cb048 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:01:21.384344 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa55e506-227c-46e4-87ae-28bf045cb048 None admin] PATH_INFO: `/v3/projects/3808a78915a44e0594da8222173d3ea4` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:01:21.385715 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa55e506-227c-46e4-87ae-28bf045cb048 None admin] RBAC: Authorizing `identity:get_project(project_id=3808a78915a44e0594da8222173d3ea4)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:01:21.387133 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa55e506-227c-46e4-87ae-28bf045cb048 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:01:21.388777 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 584/1199] 10.222.0.55 () {56 vars in 1186 bytes} [Fri Sep 23 15:01:21 2022] GET /identity/v3/projects/3808a78915a44e0594da8222173d3ea4 => generated 361 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:01:21.394959 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca53fa96-baa3-4cf4-92ca-d5b5e4d0b47e None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:01:21.486175 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:01:21.492610 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca53fa96-baa3-4cf4-92ca-d5b5e4d0b47e None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:01:21.527367 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f72db5-bf4c-407f-bae4-05337d9e266f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:01:21.528003 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f72db5-bf4c-407f-bae4-05337d9e266f None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:01:21.528484 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f72db5-bf4c-407f-bae4-05337d9e266f None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:01:21.529083 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f72db5-bf4c-407f-bae4-05337d9e266f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:01:21.529827 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f72db5-bf4c-407f-bae4-05337d9e266f None admin] RBAC: Adding query filter params (role.id=8e2919e79df040e8926f21607818b58c, scope.project.id=3808a78915a44e0594da8222173d3ea4, user.id=f49a2da9db5243c6941ab6bfeddd8a1d) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:01:21.530150 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f72db5-bf4c-407f-bae4-05337d9e266f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:01:21.531576 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f72db5-bf4c-407f-bae4-05337d9e266f None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:01:21.537565 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 616/1200] 10.222.0.55 () {56 vars in 1374 bytes} [Fri Sep 23 15:01:21 2022] GET /identity/v3/role_assignments?user.id=f49a2da9db5243c6941ab6bfeddd8a1d&scope.project.id=3808a78915a44e0594da8222173d3ea4&role.id=8e2919e79df040e8926f21607818b58c => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:01:31.379138 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=93 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:01:31.534925 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=21 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:02:44.696994 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd466e62-a039-4a28-b242-46f5f8834dff None None] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:02:44.697628 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd466e62-a039-4a28-b242-46f5f8834dff None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:02:44.697628 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd466e62-a039-4a28-b242-46f5f8834dff None None] PATH_INFO: `/` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:02:44.698866 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 585/1201] 10.222.0.55 () {54 vars in 927 bytes} [Fri Sep 23 15:02:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:02:44.704156 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36daa1cd-5dfb-4e81-8476-050b9e41cc5d None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:02:44.704643 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36daa1cd-5dfb-4e81-8476-050b9e41cc5d None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:02:44.704777 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36daa1cd-5dfb-4e81-8476-050b9e41cc5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:02:44.723364 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-36daa1cd-5dfb-4e81-8476-050b9e41cc5d None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:02:44.730322 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-36daa1cd-5dfb-4e81-8476-050b9e41cc5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:02:44.793547 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 617/1202] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Sep 23 15:02:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 23 15:02:45.065225 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd466e62-a039-4a28-b242-46f5f8834dff None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:02:45.169399 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd466e62-a039-4a28-b242-46f5f8834dff None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:02:45.202881 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370855a9-4066-4f00-be8d-37f4649e82fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:02:45.203514 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370855a9-4066-4f00-be8d-37f4649e82fc None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:02:45.203793 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370855a9-4066-4f00-be8d-37f4649e82fc None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:02:45.204102 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370855a9-4066-4f00-be8d-37f4649e82fc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:02:45.215033 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370855a9-4066-4f00-be8d-37f4649e82fc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:02:45.216513 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370855a9-4066-4f00-be8d-37f4649e82fc None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:02:45.231798 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application [None req-370855a9-4066-4f00-be8d-37f4649e82fc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 15:02:45.231798 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:02:45.231798 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:02:45.231798 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:02:45.231798 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:02:45.231798 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:02:45.231798 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:02:45.231798 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:02:45.231798 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:02:45.231798 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:02:45.231798 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:02:45.231798 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:02:45.231798 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 15:02:45.231798 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 15:02:45.231798 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 15:02:45.231798 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 15:02:45.232914 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:02:45.232914 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:02:45.232914 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:02:45.232914 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:02:45.232914 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:02:45.232914 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:02:45.232914 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:02:45.232914 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application with Lock( Sep 23 15:02:45.232914 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:02:45.232914 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:02:45.232914 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:02:45.232914 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:02:45.232914 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:02:45.232914 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:02:45.232914 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:02:45.232914 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:02:45.232914 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 15:02:45.233907 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 15:02:45.233907 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 15:02:45.233907 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Sep 23 15:02:45.233907 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 586/1203] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Sep 23 15:02:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:02:45.239256 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36daa1cd-5dfb-4e81-8476-050b9e41cc5d None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:02:45.341433 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36daa1cd-5dfb-4e81-8476-050b9e41cc5d None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:02:45.373602 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:02:45.379660 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ba7384-7a5d-4d9b-a503-2507002cc0ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:02:45.380303 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ba7384-7a5d-4d9b-a503-2507002cc0ed None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:02:45.380615 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ba7384-7a5d-4d9b-a503-2507002cc0ed None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:02:45.380890 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ba7384-7a5d-4d9b-a503-2507002cc0ed None admin] PATH_INFO: `/v3/domains` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:02:45.381564 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ba7384-7a5d-4d9b-a503-2507002cc0ed None admin] RBAC: Adding query filter params (name=Default) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:02:45.381955 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ba7384-7a5d-4d9b-a503-2507002cc0ed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:02:45.383684 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ba7384-7a5d-4d9b-a503-2507002cc0ed None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:02:45.395241 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 618/1204] 10.222.0.55 () {56 vars in 1109 bytes} [Fri Sep 23 15:02:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:02:45.401676 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370855a9-4066-4f00-be8d-37f4649e82fc None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:02:45.454226 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:02:45.505974 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370855a9-4066-4f00-be8d-37f4649e82fc None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:02:45.543872 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1f5f9f-9c6f-44cc-ad7e-5d0f0777425a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:02:45.544566 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1f5f9f-9c6f-44cc-ad7e-5d0f0777425a None admin] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:02:45.544845 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1f5f9f-9c6f-44cc-ad7e-5d0f0777425a None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:02:45.545117 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1f5f9f-9c6f-44cc-ad7e-5d0f0777425a None admin] PATH_INFO: `/v3/users` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:02:45.545784 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1f5f9f-9c6f-44cc-ad7e-5d0f0777425a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:02:45.547251 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1f5f9f-9c6f-44cc-ad7e-5d0f0777425a None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:02:45.661096 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 587/1205] 10.222.0.55 () {60 vars in 1132 bytes} [Fri Sep 23 15:02:45 2022] POST /identity/v3/users => generated 251 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:02:47.912077 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e8669c-758a-4ee5-9f38-852c2a05435d None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:02:47.913240 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e8669c-758a-4ee5-9f38-852c2a05435d None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:02:47.913240 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e8669c-758a-4ee5-9f38-852c2a05435d None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:02:47.914501 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 619/1206] 10.222.0.55 () {54 vars in 927 bytes} [Fri Sep 23 15:02:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:02:47.920522 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b046ff-656a-4b3a-b8d0-954393df4272 None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:02:47.921009 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b046ff-656a-4b3a-b8d0-954393df4272 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:02:47.921550 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b046ff-656a-4b3a-b8d0-954393df4272 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:02:47.943002 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-07b046ff-656a-4b3a-b8d0-954393df4272 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:02:47.944691 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-07b046ff-656a-4b3a-b8d0-954393df4272 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:02:48.022980 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 588/1207] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Sep 23 15:02:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 23 15:02:48.308279 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e8669c-758a-4ee5-9f38-852c2a05435d None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:02:48.420657 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e8669c-758a-4ee5-9f38-852c2a05435d None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:02:48.455189 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb7daed-0497-41ef-b839-4ae1127a6b36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:02:48.455826 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb7daed-0497-41ef-b839-4ae1127a6b36 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:02:48.456201 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb7daed-0497-41ef-b839-4ae1127a6b36 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:02:48.456480 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb7daed-0497-41ef-b839-4ae1127a6b36 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:02:48.468761 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb7daed-0497-41ef-b839-4ae1127a6b36 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 15:02:48.473722 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-9bb7daed-0497-41ef-b839-4ae1127a6b36 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 23 15:02:48.473722 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:02:48.473722 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:02:48.473722 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:02:48.473722 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:02:48.473722 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:02:48.473722 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:02:48.473722 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:02:48.473722 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:02:48.473722 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:02:48.473722 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:02:48.473722 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:02:48.473722 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 15:02:48.473722 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 15:02:48.473722 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 15:02:48.473722 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 15:02:48.478004 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 15:02:48.478004 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 15:02:48.478004 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 15:02:48.478004 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 15:02:48.478004 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:02:48.478004 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:02:48.478004 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:02:48.478004 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:02:48.478004 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:02:48.478004 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:02:48.478004 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:02:48.478004 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Sep 23 15:02:48.478004 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:02:48.478004 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:02:48.478004 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:02:48.478004 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:02:48.479159 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:02:48.479159 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:02:48.479159 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:02:48.479159 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:02:48.479159 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 15:02:48.479159 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 15:02:48.479159 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 15:02:48.479159 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 15:02:48.479159 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 15:02:48.479159 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 15:02:48.479159 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 23 15:02:48.479159 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Sep 23 15:02:48.479159 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 620/1208] 10.222.0.55 () {56 vars in 1102 bytes} [Fri Sep 23 15:02:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:02:48.481905 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b046ff-656a-4b3a-b8d0-954393df4272 None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:02:48.522991 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:02:48.587787 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b046ff-656a-4b3a-b8d0-954393df4272 None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:02:48.623498 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4983644b-cbad-460b-acbd-c543ffdb5774 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:02:48.624144 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4983644b-cbad-460b-acbd-c543ffdb5774 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:02:48.624439 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4983644b-cbad-460b-acbd-c543ffdb5774 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:02:48.624707 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4983644b-cbad-460b-acbd-c543ffdb5774 None admin] PATH_INFO: `/v3/roles` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:02:48.625377 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4983644b-cbad-460b-acbd-c543ffdb5774 None admin] RBAC: Adding query filter params (name=service) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:02:48.625684 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4983644b-cbad-460b-acbd-c543ffdb5774 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:02:48.627059 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4983644b-cbad-460b-acbd-c543ffdb5774 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:02:48.635943 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 589/1209] 10.222.0.55 () {56 vars in 1103 bytes} [Fri Sep 23 15:02:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:02:48.642178 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb7daed-0497-41ef-b839-4ae1127a6b36 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:02:48.696353 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:02:48.752517 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb7daed-0497-41ef-b839-4ae1127a6b36 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:02:48.787231 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fa4e9a-e098-4aa6-80f2-a2e83128f03b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:02:48.788095 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fa4e9a-e098-4aa6-80f2-a2e83128f03b None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:02:48.789161 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fa4e9a-e098-4aa6-80f2-a2e83128f03b None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:02:48.789516 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fa4e9a-e098-4aa6-80f2-a2e83128f03b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:02:48.802306 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fa4e9a-e098-4aa6-80f2-a2e83128f03b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:02:48.803848 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fa4e9a-e098-4aa6-80f2-a2e83128f03b None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:02:48.818211 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-58fa4e9a-e098-4aa6-80f2-a2e83128f03b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 15:02:48.818211 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:02:48.818211 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:02:48.818211 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:02:48.818211 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:02:48.818211 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:02:48.818211 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:02:48.818211 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:02:48.818211 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:02:48.818211 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:02:48.818211 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:02:48.818211 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:02:48.818211 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 15:02:48.818211 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 15:02:48.818211 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 15:02:48.818211 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 15:02:48.819756 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:02:48.819756 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:02:48.819756 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:02:48.819756 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:02:48.819756 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:02:48.819756 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:02:48.819756 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:02:48.819756 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Sep 23 15:02:48.819756 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:02:48.819756 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:02:48.819756 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:02:48.819756 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:02:48.819756 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:02:48.819756 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:02:48.819756 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:02:48.819756 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:02:48.819756 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 15:02:48.820745 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 15:02:48.820745 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 15:02:48.820745 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Sep 23 15:02:48.820745 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 621/1210] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Sep 23 15:02:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:02:48.826027 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4983644b-cbad-460b-acbd-c543ffdb5774 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:02:48.931223 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:02:48.934044 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4983644b-cbad-460b-acbd-c543ffdb5774 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:02:48.969425 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3de2a04-71b5-4921-ba72-bd2fb1e6c107 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:02:48.970043 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3de2a04-71b5-4921-ba72-bd2fb1e6c107 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:02:48.970320 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3de2a04-71b5-4921-ba72-bd2fb1e6c107 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:02:48.970592 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3de2a04-71b5-4921-ba72-bd2fb1e6c107 None admin] PATH_INFO: `/v3/domains` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:02:48.971240 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3de2a04-71b5-4921-ba72-bd2fb1e6c107 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:02:48.971537 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3de2a04-71b5-4921-ba72-bd2fb1e6c107 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:02:48.973030 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3de2a04-71b5-4921-ba72-bd2fb1e6c107 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:02:48.985597 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 590/1211] 10.222.0.55 () {56 vars in 1109 bytes} [Fri Sep 23 15:02:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:02:48.993037 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fa4e9a-e098-4aa6-80f2-a2e83128f03b None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:02:49.102426 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fa4e9a-e098-4aa6-80f2-a2e83128f03b None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:02:49.104047 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:02:49.139406 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5703617f-69de-4268-a7cb-14645dad7ab3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:02:49.140074 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5703617f-69de-4268-a7cb-14645dad7ab3 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:02:49.140358 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5703617f-69de-4268-a7cb-14645dad7ab3 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:02:49.140631 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5703617f-69de-4268-a7cb-14645dad7ab3 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:02:49.149029 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5703617f-69de-4268-a7cb-14645dad7ab3 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:02:49.150509 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5703617f-69de-4268-a7cb-14645dad7ab3 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:02:49.162229 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-5703617f-69de-4268-a7cb-14645dad7ab3 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 23 15:02:49.162229 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:02:49.162229 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:02:49.162229 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:02:49.162229 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:02:49.162229 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:02:49.162229 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:02:49.162229 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:02:49.162229 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:02:49.162229 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:02:49.162229 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:02:49.162229 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:02:49.162229 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 23 15:02:49.162229 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 23 15:02:49.162229 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 23 15:02:49.162229 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 23 15:02:49.163402 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:02:49.163402 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:02:49.163402 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 23 15:02:49.163402 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 15:02:49.163402 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 23 15:02:49.163402 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 15:02:49.163402 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:02:49.163402 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:02:49.163402 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:02:49.163402 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:02:49.163402 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:02:49.163402 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Sep 23 15:02:49.163402 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:02:49.163402 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:02:49.163402 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:02:49.163402 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:02:49.163402 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:02:49.163402 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:02:49.164682 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:02:49.164682 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:02:49.164682 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 23 15:02:49.164682 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 23 15:02:49.164682 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 23 15:02:49.164682 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 23 15:02:49.164682 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 23 15:02:49.164682 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 23 15:02:49.164682 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 23 15:02:49.164682 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Sep 23 15:02:49.164682 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 622/1212] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Sep 23 15:02:48 2022] GET /identity/v3/users/placement => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:02:49.169803 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3de2a04-71b5-4921-ba72-bd2fb1e6c107 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:02:49.276570 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3de2a04-71b5-4921-ba72-bd2fb1e6c107 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:02:49.313600 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f40988-8460-4a59-811b-f29328b46098 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:02:49.314209 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f40988-8460-4a59-811b-f29328b46098 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:02:49.314531 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f40988-8460-4a59-811b-f29328b46098 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:02:49.314805 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f40988-8460-4a59-811b-f29328b46098 None admin] PATH_INFO: `/v3/users` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:02:49.315527 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f40988-8460-4a59-811b-f29328b46098 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:02:49.315885 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f40988-8460-4a59-811b-f29328b46098 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:02:49.317353 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f40988-8460-4a59-811b-f29328b46098 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:02:49.337308 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 591/1213] 10.222.0.55 () {56 vars in 1143 bytes} [Fri Sep 23 15:02:49 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:02:49.343279 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5703617f-69de-4268-a7cb-14645dad7ab3 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:02:49.457063 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5703617f-69de-4268-a7cb-14645dad7ab3 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:02:49.493277 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315122cd-a80c-431c-8542-29234a8f0d5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:02:49.493886 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315122cd-a80c-431c-8542-29234a8f0d5e None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:02:49.494168 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315122cd-a80c-431c-8542-29234a8f0d5e None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:02:49.494443 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315122cd-a80c-431c-8542-29234a8f0d5e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:02:49.526164 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:02:49.564784 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315122cd-a80c-431c-8542-29234a8f0d5e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:02:49.566154 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315122cd-a80c-431c-8542-29234a8f0d5e None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:02:49.670104 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 623/1214] 10.222.0.55 () {58 vars in 1282 bytes} [Fri Sep 23 15:02:49 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 23 15:02:49.676712 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f40988-8460-4a59-811b-f29328b46098 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:02:49.698714 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:02:49.782011 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f40988-8460-4a59-811b-f29328b46098 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:02:49.822510 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44ee623-cb08-4e60-b8e5-e6e5e834bf82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:02:49.823064 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44ee623-cb08-4e60-b8e5-e6e5e834bf82 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:02:49.823346 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44ee623-cb08-4e60-b8e5-e6e5e834bf82 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:02:49.823621 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44ee623-cb08-4e60-b8e5-e6e5e834bf82 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:02:49.834383 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44ee623-cb08-4e60-b8e5-e6e5e834bf82 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:02:49.836413 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44ee623-cb08-4e60-b8e5-e6e5e834bf82 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:02:49.853392 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application [None req-f44ee623-cb08-4e60-b8e5-e6e5e834bf82 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 15:02:49.853392 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:02:49.853392 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:02:49.853392 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:02:49.853392 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:02:49.853392 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:02:49.853392 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:02:49.853392 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:02:49.853392 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:02:49.853392 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:02:49.853392 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:02:49.853392 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:02:49.853392 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 15:02:49.853392 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 15:02:49.853392 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 15:02:49.853392 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 15:02:49.854599 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:02:49.854599 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:02:49.854599 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:02:49.854599 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:02:49.854599 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:02:49.854599 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:02:49.854599 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:02:49.854599 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application with Lock( Sep 23 15:02:49.854599 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:02:49.854599 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:02:49.854599 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:02:49.854599 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:02:49.854599 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:02:49.854599 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:02:49.854599 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:02:49.854599 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:02:49.854599 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 15:02:49.855647 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 15:02:49.855647 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 15:02:49.855647 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Sep 23 15:02:49.855647 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 592/1215] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Sep 23 15:02:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:02:49.860595 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315122cd-a80c-431c-8542-29234a8f0d5e None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:02:49.926096 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:02:49.963720 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315122cd-a80c-431c-8542-29234a8f0d5e None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:02:49.998022 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91edc1db-3b13-4781-984e-d6a7798d49d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:02:49.998744 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91edc1db-3b13-4781-984e-d6a7798d49d4 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:02:49.998902 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91edc1db-3b13-4781-984e-d6a7798d49d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:02:49.999171 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91edc1db-3b13-4781-984e-d6a7798d49d4 None admin] PATH_INFO: `/v3/domains` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:02:49.999811 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91edc1db-3b13-4781-984e-d6a7798d49d4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:02:50.000139 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91edc1db-3b13-4781-984e-d6a7798d49d4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:02:50.001500 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91edc1db-3b13-4781-984e-d6a7798d49d4 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:02:50.013473 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 624/1216] 10.222.0.55 () {56 vars in 1109 bytes} [Fri Sep 23 15:02:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 15:02:50.019503 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44ee623-cb08-4e60-b8e5-e6e5e834bf82 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:02:50.095331 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:02:50.127892 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44ee623-cb08-4e60-b8e5-e6e5e834bf82 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:02:50.165717 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491d7c1e-6221-44b4-af9a-8ce669a5ac6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:02:50.166356 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491d7c1e-6221-44b4-af9a-8ce669a5ac6b None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:02:50.166647 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491d7c1e-6221-44b4-af9a-8ce669a5ac6b None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:02:50.166908 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491d7c1e-6221-44b4-af9a-8ce669a5ac6b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:02:50.174210 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491d7c1e-6221-44b4-af9a-8ce669a5ac6b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:02:50.175655 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491d7c1e-6221-44b4-af9a-8ce669a5ac6b None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:02:50.187240 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application [None req-491d7c1e-6221-44b4-af9a-8ce669a5ac6b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 15:02:50.187240 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:02:50.187240 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:02:50.187240 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:02:50.187240 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:02:50.187240 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:02:50.187240 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:02:50.187240 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:02:50.187240 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:02:50.187240 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:02:50.187240 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:02:50.187240 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:02:50.187240 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 23 15:02:50.187240 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 23 15:02:50.187240 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 23 15:02:50.193320 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 23 15:02:50.193320 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:02:50.193320 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:02:50.193320 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:02:50.193320 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:02:50.193320 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:02:50.193320 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:02:50.193320 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:02:50.193320 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application with Lock( Sep 23 15:02:50.193320 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:02:50.193320 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:02:50.193320 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:02:50.193320 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:02:50.193320 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:02:50.193320 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:02:50.193320 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:02:50.193320 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:02:50.195099 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 23 15:02:50.195099 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 23 15:02:50.195099 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 23 15:02:50.195099 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 23 15:02:50.195099 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 23 15:02:50.195099 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 23 15:02:50.195099 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 15:02:50.195099 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Sep 23 15:02:50.195099 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 593/1217] 10.222.0.55 () {56 vars in 1111 bytes} [Fri Sep 23 15:02:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:02:50.195762 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91edc1db-3b13-4781-984e-d6a7798d49d4 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:02:50.302955 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91edc1db-3b13-4781-984e-d6a7798d49d4 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:02:50.326567 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:02:50.341244 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0668998-3d86-47f6-ab5b-1457ba4ba252 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:02:50.341898 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0668998-3d86-47f6-ab5b-1457ba4ba252 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:02:50.342220 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0668998-3d86-47f6-ab5b-1457ba4ba252 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:02:50.342534 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0668998-3d86-47f6-ab5b-1457ba4ba252 None admin] PATH_INFO: `/v3/projects` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:02:50.343277 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0668998-3d86-47f6-ab5b-1457ba4ba252 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:02:50.343626 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0668998-3d86-47f6-ab5b-1457ba4ba252 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:02:50.345470 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0668998-3d86-47f6-ab5b-1457ba4ba252 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:02:50.358332 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 625/1218] 10.222.0.55 () {56 vars in 1148 bytes} [Fri Sep 23 15:02:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:02:50.364582 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491d7c1e-6221-44b4-af9a-8ce669a5ac6b None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:02:50.470477 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491d7c1e-6221-44b4-af9a-8ce669a5ac6b None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:02:50.500316 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:02:50.509355 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252db8ec-5fd0-422e-9360-dd43e07a88b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:02:50.509972 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252db8ec-5fd0-422e-9360-dd43e07a88b8 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:02:50.510359 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252db8ec-5fd0-422e-9360-dd43e07a88b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:02:50.510656 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252db8ec-5fd0-422e-9360-dd43e07a88b8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:02:50.511885 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252db8ec-5fd0-422e-9360-dd43e07a88b8 None admin] RBAC: Adding query filter params (role.id=0665869b71fa40b0b552efdd57faf023, scope.project.id=87b3e9d4b6474bd4a04ba59ef52bf3e2, user.id=09f1a55d180642c781a6e0b017d9f0aa) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:02:50.512249 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252db8ec-5fd0-422e-9360-dd43e07a88b8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:02:50.513700 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252db8ec-5fd0-422e-9360-dd43e07a88b8 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:02:50.520209 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 594/1219] 10.222.0.55 () {56 vars in 1374 bytes} [Fri Sep 23 15:02:50 2022] GET /identity/v3/role_assignments?user.id=09f1a55d180642c781a6e0b017d9f0aa&scope.project.id=87b3e9d4b6474bd4a04ba59ef52bf3e2&role.id=0665869b71fa40b0b552efdd57faf023 => generated 259 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:02:52.718081 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03666014-2614-4603-a1e0-efcd922f00e6 None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:02:52.718721 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03666014-2614-4603-a1e0-efcd922f00e6 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:02:52.718721 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03666014-2614-4603-a1e0-efcd922f00e6 None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:02:52.720037 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 626/1220] 10.222.0.55 () {54 vars in 927 bytes} [Fri Sep 23 15:02:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:02:52.726040 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d8d91a-85e1-4863-8a27-cb1caac1aae5 None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:02:52.726576 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d8d91a-85e1-4863-8a27-cb1caac1aae5 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:02:52.726576 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d8d91a-85e1-4863-8a27-cb1caac1aae5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:02:52.745698 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-b5d8d91a-85e1-4863-8a27-cb1caac1aae5 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:02:52.746908 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-b5d8d91a-85e1-4863-8a27-cb1caac1aae5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:02:52.819858 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 595/1221] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Sep 23 15:02:52 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 23 15:02:53.087743 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03666014-2614-4603-a1e0-efcd922f00e6 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:02:53.193192 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03666014-2614-4603-a1e0-efcd922f00e6 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:02:53.230263 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2772e9fe-1ac3-43fc-ab66-6cc85e8a7b3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:02:53.235435 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2772e9fe-1ac3-43fc-ab66-6cc85e8a7b3f None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:02:53.235435 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2772e9fe-1ac3-43fc-ab66-6cc85e8a7b3f None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:02:53.235435 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2772e9fe-1ac3-43fc-ab66-6cc85e8a7b3f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:02:53.242449 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2772e9fe-1ac3-43fc-ab66-6cc85e8a7b3f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 15:02:53.247302 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-2772e9fe-1ac3-43fc-ab66-6cc85e8a7b3f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 23 15:02:53.247302 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:02:53.247302 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:02:53.247302 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:02:53.247302 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:02:53.247302 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:02:53.247302 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:02:53.247302 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:02:53.247302 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:02:53.247302 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:02:53.247302 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:02:53.247302 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:02:53.247302 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 15:02:53.247302 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 15:02:53.247302 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 15:02:53.247302 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 15:02:53.248497 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 15:02:53.248497 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 15:02:53.248497 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 15:02:53.248497 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 15:02:53.248497 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:02:53.248497 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:02:53.248497 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:02:53.248497 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:02:53.248497 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:02:53.248497 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:02:53.248497 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:02:53.248497 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Sep 23 15:02:53.248497 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:02:53.248497 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:02:53.248497 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:02:53.248497 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:02:53.249439 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:02:53.249439 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:02:53.249439 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:02:53.249439 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:02:53.249439 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 15:02:53.249439 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 15:02:53.249439 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 15:02:53.249439 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 15:02:53.249439 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 15:02:53.249439 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 15:02:53.249439 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 23 15:02:53.249439 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Sep 23 15:02:53.249439 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 627/1222] 10.222.0.55 () {56 vars in 1102 bytes} [Fri Sep 23 15:02:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:02:53.254658 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d8d91a-85e1-4863-8a27-cb1caac1aae5 None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:02:53.358527 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d8d91a-85e1-4863-8a27-cb1caac1aae5 None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:02:53.384308 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:02:53.394652 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf62659d-4e5e-4e0b-9847-628c57a84e91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:02:53.395270 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf62659d-4e5e-4e0b-9847-628c57a84e91 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:02:53.395549 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf62659d-4e5e-4e0b-9847-628c57a84e91 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:02:53.395900 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf62659d-4e5e-4e0b-9847-628c57a84e91 None admin] PATH_INFO: `/v3/roles` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:02:53.396590 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf62659d-4e5e-4e0b-9847-628c57a84e91 None admin] RBAC: Adding query filter params (name=service) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:02:53.397494 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf62659d-4e5e-4e0b-9847-628c57a84e91 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:02:53.398863 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf62659d-4e5e-4e0b-9847-628c57a84e91 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:02:53.406871 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 596/1223] 10.222.0.55 () {56 vars in 1103 bytes} [Fri Sep 23 15:02:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:02:53.413209 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2772e9fe-1ac3-43fc-ab66-6cc85e8a7b3f None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:02:53.464828 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:02:53.524613 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2772e9fe-1ac3-43fc-ab66-6cc85e8a7b3f None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:02:53.560753 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb84ea96-e33b-4501-a7b5-2436c8891cde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:02:53.561385 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb84ea96-e33b-4501-a7b5-2436c8891cde None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:02:53.561671 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb84ea96-e33b-4501-a7b5-2436c8891cde None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:02:53.561999 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb84ea96-e33b-4501-a7b5-2436c8891cde None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:02:53.573170 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb84ea96-e33b-4501-a7b5-2436c8891cde None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:02:53.574717 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb84ea96-e33b-4501-a7b5-2436c8891cde None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:02:53.590970 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-bb84ea96-e33b-4501-a7b5-2436c8891cde None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 15:02:53.590970 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:02:53.590970 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:02:53.590970 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:02:53.590970 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:02:53.590970 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:02:53.590970 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:02:53.590970 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:02:53.590970 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:02:53.590970 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:02:53.590970 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:02:53.590970 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:02:53.590970 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 15:02:53.590970 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 15:02:53.590970 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 15:02:53.590970 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 15:02:53.594536 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:02:53.594536 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:02:53.594536 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:02:53.594536 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:02:53.594536 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:02:53.594536 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:02:53.594536 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:02:53.594536 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Sep 23 15:02:53.594536 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:02:53.594536 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:02:53.594536 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:02:53.594536 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:02:53.594536 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:02:53.594536 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:02:53.594536 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:02:53.594536 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:02:53.594536 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 15:02:53.595652 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 15:02:53.595652 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 15:02:53.595652 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Sep 23 15:02:53.595652 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 628/1224] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Sep 23 15:02:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:02:53.597916 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf62659d-4e5e-4e0b-9847-628c57a84e91 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:02:53.704705 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf62659d-4e5e-4e0b-9847-628c57a84e91 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:02:53.743584 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3f779a-3275-4a01-8304-753a0093c319 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:02:53.744293 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3f779a-3275-4a01-8304-753a0093c319 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:02:53.744487 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3f779a-3275-4a01-8304-753a0093c319 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:02:53.744756 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3f779a-3275-4a01-8304-753a0093c319 None admin] PATH_INFO: `/v3/domains` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:02:53.745390 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3f779a-3275-4a01-8304-753a0093c319 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:02:53.745692 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3f779a-3275-4a01-8304-753a0093c319 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:02:53.747052 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3f779a-3275-4a01-8304-753a0093c319 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:02:53.761441 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 597/1225] 10.222.0.55 () {56 vars in 1109 bytes} [Fri Sep 23 15:02:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:02:53.767312 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb84ea96-e33b-4501-a7b5-2436c8891cde None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:02:53.863839 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:02:53.872766 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb84ea96-e33b-4501-a7b5-2436c8891cde None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:02:53.906292 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6808c9d9-18d7-45f4-ad6e-cfaef4974dd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:02:53.906913 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6808c9d9-18d7-45f4-ad6e-cfaef4974dd7 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:02:53.907791 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6808c9d9-18d7-45f4-ad6e-cfaef4974dd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:02:53.907791 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6808c9d9-18d7-45f4-ad6e-cfaef4974dd7 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:02:53.915635 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6808c9d9-18d7-45f4-ad6e-cfaef4974dd7 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:02:53.917114 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6808c9d9-18d7-45f4-ad6e-cfaef4974dd7 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:02:53.929985 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-6808c9d9-18d7-45f4-ad6e-cfaef4974dd7 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 23 15:02:53.929985 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:02:53.929985 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:02:53.929985 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:02:53.929985 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:02:53.929985 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:02:53.929985 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:02:53.929985 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:02:53.929985 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:02:53.929985 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:02:53.929985 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:02:53.929985 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:02:53.929985 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 23 15:02:53.929985 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 23 15:02:53.929985 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 23 15:02:53.929985 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 23 15:02:53.931808 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:02:53.931808 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:02:53.931808 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 23 15:02:53.931808 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 15:02:53.931808 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 23 15:02:53.931808 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 15:02:53.931808 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:02:53.931808 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:02:53.931808 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:02:53.931808 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:02:53.931808 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:02:53.931808 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Sep 23 15:02:53.931808 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:02:53.931808 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:02:53.931808 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:02:53.931808 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:02:53.931808 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:02:53.931808 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:02:53.932951 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:02:53.932951 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:02:53.932951 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 23 15:02:53.932951 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 23 15:02:53.932951 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 23 15:02:53.932951 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 23 15:02:53.932951 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 23 15:02:53.932951 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 23 15:02:53.932951 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 23 15:02:53.932951 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Sep 23 15:02:53.932951 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 629/1226] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Sep 23 15:02:53 2022] GET /identity/v3/users/placement => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:02:53.937179 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3f779a-3275-4a01-8304-753a0093c319 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:02:53.985688 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:02:54.049053 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3f779a-3275-4a01-8304-753a0093c319 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:02:54.088468 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acfbdea-b812-4187-84e6-6c614c10e22b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:02:54.089085 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acfbdea-b812-4187-84e6-6c614c10e22b None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:02:54.089369 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acfbdea-b812-4187-84e6-6c614c10e22b None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:02:54.089643 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acfbdea-b812-4187-84e6-6c614c10e22b None admin] PATH_INFO: `/v3/users` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:02:54.090372 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acfbdea-b812-4187-84e6-6c614c10e22b None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:02:54.090676 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acfbdea-b812-4187-84e6-6c614c10e22b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:02:54.092222 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acfbdea-b812-4187-84e6-6c614c10e22b None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:02:54.111819 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 598/1227] 10.222.0.55 () {56 vars in 1143 bytes} [Fri Sep 23 15:02:53 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 15:02:54.118489 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6808c9d9-18d7-45f4-ad6e-cfaef4974dd7 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:02:54.232986 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6808c9d9-18d7-45f4-ad6e-cfaef4974dd7 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:02:54.267373 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8facc921-48d0-4cf6-9816-6644b75a1b1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:02:54.267926 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8facc921-48d0-4cf6-9816-6644b75a1b1d None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:02:54.268237 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8facc921-48d0-4cf6-9816-6644b75a1b1d None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:02:54.278429 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8facc921-48d0-4cf6-9816-6644b75a1b1d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:02:54.279626 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8facc921-48d0-4cf6-9816-6644b75a1b1d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:02:54.281206 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8facc921-48d0-4cf6-9816-6644b75a1b1d None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:02:54.297157 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-8facc921-48d0-4cf6-9816-6644b75a1b1d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 15:02:54.297157 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:02:54.297157 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:02:54.297157 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:02:54.297157 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:02:54.297157 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:02:54.297157 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:02:54.297157 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:02:54.297157 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:02:54.297157 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:02:54.297157 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:02:54.297157 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:02:54.297157 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 15:02:54.297157 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 15:02:54.297157 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 15:02:54.297157 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 15:02:54.298462 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:02:54.298462 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:02:54.298462 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:02:54.298462 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:02:54.298462 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:02:54.298462 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:02:54.298462 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:02:54.298462 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Sep 23 15:02:54.298462 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:02:54.298462 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:02:54.298462 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:02:54.298462 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:02:54.298462 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:02:54.298462 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:02:54.298462 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:02:54.298462 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:02:54.298462 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 15:02:54.299379 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 15:02:54.299379 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 15:02:54.299379 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Sep 23 15:02:54.299379 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 630/1228] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Sep 23 15:02:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:02:54.304690 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acfbdea-b812-4187-84e6-6c614c10e22b None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:02:54.410699 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acfbdea-b812-4187-84e6-6c614c10e22b None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:02:54.413632 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:02:54.446695 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88916648-d28b-4959-a3cd-735fbdd77508 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:02:54.447369 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88916648-d28b-4959-a3cd-735fbdd77508 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:02:54.447647 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88916648-d28b-4959-a3cd-735fbdd77508 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:02:54.447921 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88916648-d28b-4959-a3cd-735fbdd77508 None admin] PATH_INFO: `/v3/domains` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:02:54.448596 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88916648-d28b-4959-a3cd-735fbdd77508 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:02:54.448895 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88916648-d28b-4959-a3cd-735fbdd77508 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:02:54.450254 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88916648-d28b-4959-a3cd-735fbdd77508 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:02:54.462060 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 599/1229] 10.222.0.55 () {56 vars in 1109 bytes} [Fri Sep 23 15:02:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 15:02:54.470400 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8facc921-48d0-4cf6-9816-6644b75a1b1d None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:02:54.482567 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:02:54.587518 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8facc921-48d0-4cf6-9816-6644b75a1b1d None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:02:54.622846 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aaf9a78-a0d5-48ff-9507-cf5707753788 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:02:54.623493 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaf9a78-a0d5-48ff-9507-cf5707753788 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:02:54.623872 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaf9a78-a0d5-48ff-9507-cf5707753788 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:02:54.624174 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaf9a78-a0d5-48ff-9507-cf5707753788 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:02:54.630703 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aaf9a78-a0d5-48ff-9507-cf5707753788 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:02:54.632234 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aaf9a78-a0d5-48ff-9507-cf5707753788 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:02:54.643055 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-8aaf9a78-a0d5-48ff-9507-cf5707753788 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 15:02:54.643055 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:02:54.643055 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:02:54.643055 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:02:54.643055 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:02:54.643055 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:02:54.643055 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:02:54.643055 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:02:54.643055 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:02:54.643055 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:02:54.643055 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:02:54.643055 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:02:54.643055 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 23 15:02:54.643055 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 23 15:02:54.643055 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 23 15:02:54.644154 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 23 15:02:54.644154 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:02:54.644154 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:02:54.644154 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:02:54.644154 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:02:54.644154 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:02:54.644154 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:02:54.644154 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:02:54.644154 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Sep 23 15:02:54.644154 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:02:54.644154 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:02:54.644154 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:02:54.644154 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:02:54.644154 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:02:54.644154 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:02:54.644154 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:02:54.644154 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:02:54.645299 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 23 15:02:54.645299 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 23 15:02:54.645299 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 23 15:02:54.645299 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 23 15:02:54.645299 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 23 15:02:54.645299 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 23 15:02:54.645299 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 15:02:54.645299 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Sep 23 15:02:54.645299 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 631/1230] 10.222.0.55 () {56 vars in 1111 bytes} [Fri Sep 23 15:02:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 23 15:02:54.650707 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88916648-d28b-4959-a3cd-735fbdd77508 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:02:54.757745 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88916648-d28b-4959-a3cd-735fbdd77508 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:02:54.792505 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ff1dcf-66cf-45e7-a83a-c1ad3788f2fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:02:54.793109 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ff1dcf-66cf-45e7-a83a-c1ad3788f2fb None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:02:54.793391 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ff1dcf-66cf-45e7-a83a-c1ad3788f2fb None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:02:54.793663 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ff1dcf-66cf-45e7-a83a-c1ad3788f2fb None admin] PATH_INFO: `/v3/projects` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:02:54.794353 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ff1dcf-66cf-45e7-a83a-c1ad3788f2fb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:02:54.794656 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ff1dcf-66cf-45e7-a83a-c1ad3788f2fb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:02:54.796103 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ff1dcf-66cf-45e7-a83a-c1ad3788f2fb None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:02:54.807783 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 600/1231] 10.222.0.55 () {56 vars in 1148 bytes} [Fri Sep 23 15:02:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:02:54.814070 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aaf9a78-a0d5-48ff-9507-cf5707753788 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:02:54.885769 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:02:54.926548 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aaf9a78-a0d5-48ff-9507-cf5707753788 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:02:54.960607 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e666577c-3b81-4258-93e5-fee354931202 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:02:54.961268 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e666577c-3b81-4258-93e5-fee354931202 None admin] REQUEST_METHOD: `PUT` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:02:54.961543 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e666577c-3b81-4258-93e5-fee354931202 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:02:54.961815 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e666577c-3b81-4258-93e5-fee354931202 None admin] PATH_INFO: `/v3/projects/87b3e9d4b6474bd4a04ba59ef52bf3e2/users/09f1a55d180642c781a6e0b017d9f0aa/roles/0665869b71fa40b0b552efdd57faf023` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:02:54.975761 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e666577c-3b81-4258-93e5-fee354931202 None admin] RBAC: Authorizing `identity:create_grant(project_id=87b3e9d4b6474bd4a04ba59ef52bf3e2, user_id=09f1a55d180642c781a6e0b017d9f0aa, role_id=0665869b71fa40b0b552efdd57faf023)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:02:54.977302 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e666577c-3b81-4258-93e5-fee354931202 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:02:55.065500 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 632/1232] 10.222.0.55 () {58 vars in 1439 bytes} [Fri Sep 23 15:02:54 2022] PUT /identity/v3/projects/87b3e9d4b6474bd4a04ba59ef52bf3e2/users/09f1a55d180642c781a6e0b017d9f0aa/roles/0665869b71fa40b0b552efdd57faf023 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:02:57.283643 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86fdc57-aea7-4135-93f4-5762b801266a None None] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:02:57.283643 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86fdc57-aea7-4135-93f4-5762b801266a None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:02:57.283643 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86fdc57-aea7-4135-93f4-5762b801266a None None] PATH_INFO: `/` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:02:57.284496 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 601/1233] 10.222.0.55 () {54 vars in 927 bytes} [Fri Sep 23 15:02:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:02:57.290393 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36acc0d9-797f-40fa-8e58-07961cd028e1 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:02:57.290863 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36acc0d9-797f-40fa-8e58-07961cd028e1 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:02:57.290957 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36acc0d9-797f-40fa-8e58-07961cd028e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:02:57.313098 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-36acc0d9-797f-40fa-8e58-07961cd028e1 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:02:57.314301 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-36acc0d9-797f-40fa-8e58-07961cd028e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:02:57.387996 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 633/1234] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Sep 23 15:02:57 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 23 15:02:57.656081 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86fdc57-aea7-4135-93f4-5762b801266a None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:02:57.683938 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:02:57.763438 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86fdc57-aea7-4135-93f4-5762b801266a None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:02:57.800780 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9557ba-9c3e-4f39-b420-4e671099f933 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:02:57.801447 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9557ba-9c3e-4f39-b420-4e671099f933 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:02:57.801730 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9557ba-9c3e-4f39-b420-4e671099f933 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:02:57.802002 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9557ba-9c3e-4f39-b420-4e671099f933 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:02:57.812730 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9557ba-9c3e-4f39-b420-4e671099f933 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 15:02:57.817485 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application [None req-cd9557ba-9c3e-4f39-b420-4e671099f933 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 23 15:02:57.817485 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:02:57.817485 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:02:57.817485 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:02:57.817485 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:02:57.817485 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:02:57.817485 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:02:57.817485 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:02:57.817485 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:02:57.817485 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:02:57.817485 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:02:57.817485 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:02:57.817485 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 15:02:57.817485 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 15:02:57.817485 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 15:02:57.817485 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 15:02:57.820095 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 15:02:57.820095 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 15:02:57.820095 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 15:02:57.820095 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 15:02:57.820095 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:02:57.820095 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:02:57.820095 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:02:57.820095 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:02:57.820095 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:02:57.820095 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:02:57.820095 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:02:57.820095 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application with Lock( Sep 23 15:02:57.820095 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:02:57.820095 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:02:57.820095 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:02:57.820095 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:02:57.820965 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:02:57.820965 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:02:57.820965 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:02:57.820965 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:02:57.820965 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 15:02:57.820965 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 15:02:57.820965 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 15:02:57.820965 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 15:02:57.820965 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 15:02:57.820965 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 15:02:57.820965 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 23 15:02:57.820965 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Sep 23 15:02:57.820965 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 602/1235] 10.222.0.55 () {56 vars in 1102 bytes} [Fri Sep 23 15:02:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:02:57.824951 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36acc0d9-797f-40fa-8e58-07961cd028e1 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:02:57.873953 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:02:57.928873 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36acc0d9-797f-40fa-8e58-07961cd028e1 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:02:57.964750 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d842e7d0-2a88-44cb-aa6c-447e7047174c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:02:57.965370 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d842e7d0-2a88-44cb-aa6c-447e7047174c None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:02:57.965655 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d842e7d0-2a88-44cb-aa6c-447e7047174c None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:02:57.965930 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d842e7d0-2a88-44cb-aa6c-447e7047174c None admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:02:57.966601 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d842e7d0-2a88-44cb-aa6c-447e7047174c None admin] RBAC: Adding query filter params (name=service) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:02:57.966907 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d842e7d0-2a88-44cb-aa6c-447e7047174c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:02:57.968294 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d842e7d0-2a88-44cb-aa6c-447e7047174c None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:02:57.977840 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 634/1236] 10.222.0.55 () {56 vars in 1103 bytes} [Fri Sep 23 15:02:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 15:02:57.984325 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9557ba-9c3e-4f39-b420-4e671099f933 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:02:58.079902 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:02:58.098163 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9557ba-9c3e-4f39-b420-4e671099f933 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:02:58.133805 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bcdbbd-703a-436e-9f61-132d10c397fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:02:58.134504 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bcdbbd-703a-436e-9f61-132d10c397fc None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:02:58.134697 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bcdbbd-703a-436e-9f61-132d10c397fc None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:02:58.134965 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bcdbbd-703a-436e-9f61-132d10c397fc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:02:58.146016 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bcdbbd-703a-436e-9f61-132d10c397fc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:02:58.148055 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bcdbbd-703a-436e-9f61-132d10c397fc None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:02:58.164278 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application [None req-06bcdbbd-703a-436e-9f61-132d10c397fc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 15:02:58.164278 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:02:58.164278 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:02:58.164278 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:02:58.164278 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:02:58.164278 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:02:58.164278 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:02:58.164278 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:02:58.164278 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:02:58.164278 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:02:58.164278 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:02:58.164278 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:02:58.164278 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 15:02:58.164278 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 15:02:58.164278 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 15:02:58.164278 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 15:02:58.172206 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:02:58.172206 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:02:58.172206 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:02:58.172206 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:02:58.172206 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:02:58.172206 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:02:58.172206 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:02:58.172206 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application with Lock( Sep 23 15:02:58.172206 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:02:58.172206 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:02:58.172206 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:02:58.172206 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:02:58.172206 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:02:58.172206 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:02:58.172206 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:02:58.172206 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:02:58.172206 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 15:02:58.173353 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 15:02:58.173353 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 15:02:58.173353 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Sep 23 15:02:58.173353 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 603/1237] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Sep 23 15:02:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:02:58.173584 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d842e7d0-2a88-44cb-aa6c-447e7047174c None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:02:58.276274 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d842e7d0-2a88-44cb-aa6c-447e7047174c None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:02:58.285860 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:02:58.312082 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eaf32cb-ca33-428e-92aa-7f61873966ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:02:58.312757 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaf32cb-ca33-428e-92aa-7f61873966ac None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:02:58.312941 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaf32cb-ca33-428e-92aa-7f61873966ac None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:02:58.313320 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaf32cb-ca33-428e-92aa-7f61873966ac None admin] PATH_INFO: `/v3/domains` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:02:58.313963 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eaf32cb-ca33-428e-92aa-7f61873966ac None admin] RBAC: Adding query filter params (name=Default) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:02:58.314277 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eaf32cb-ca33-428e-92aa-7f61873966ac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:02:58.315716 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eaf32cb-ca33-428e-92aa-7f61873966ac None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:02:58.328569 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 635/1238] 10.222.0.55 () {56 vars in 1109 bytes} [Fri Sep 23 15:02:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:02:58.335005 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bcdbbd-703a-436e-9f61-132d10c397fc None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:02:58.443228 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bcdbbd-703a-436e-9f61-132d10c397fc None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:02:58.479759 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e3b0dd-677e-4752-906e-56311ea5a9c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:02:58.480428 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e3b0dd-677e-4752-906e-56311ea5a9c1 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:02:58.480709 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e3b0dd-677e-4752-906e-56311ea5a9c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:02:58.480983 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e3b0dd-677e-4752-906e-56311ea5a9c1 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:02:58.483747 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:02:58.488906 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e3b0dd-677e-4752-906e-56311ea5a9c1 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:02:58.490357 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e3b0dd-677e-4752-906e-56311ea5a9c1 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:02:58.501570 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application [None req-43e3b0dd-677e-4752-906e-56311ea5a9c1 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 23 15:02:58.501570 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:02:58.501570 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:02:58.501570 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:02:58.501570 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:02:58.501570 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:02:58.501570 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:02:58.501570 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:02:58.501570 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:02:58.501570 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:02:58.501570 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:02:58.501570 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:02:58.501570 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 23 15:02:58.501570 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 23 15:02:58.501570 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 23 15:02:58.501570 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 23 15:02:58.502801 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:02:58.502801 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:02:58.502801 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 23 15:02:58.502801 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 15:02:58.502801 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 23 15:02:58.502801 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 15:02:58.502801 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:02:58.502801 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:02:58.502801 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:02:58.502801 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:02:58.502801 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:02:58.502801 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application with Lock( Sep 23 15:02:58.502801 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:02:58.502801 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:02:58.502801 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:02:58.502801 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:02:58.502801 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:02:58.502801 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:02:58.504075 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:02:58.504075 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:02:58.504075 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 23 15:02:58.504075 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 23 15:02:58.504075 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 23 15:02:58.504075 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 23 15:02:58.504075 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 23 15:02:58.504075 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 23 15:02:58.504075 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 23 15:02:58.504075 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Sep 23 15:02:58.504075 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 604/1239] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Sep 23 15:02:58 2022] GET /identity/v3/users/placement => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:02:58.508996 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eaf32cb-ca33-428e-92aa-7f61873966ac None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:02:58.618226 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eaf32cb-ca33-428e-92aa-7f61873966ac None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:02:58.652738 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b58401f-ddb0-4388-8c6a-4280593474db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:02:58.653342 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b58401f-ddb0-4388-8c6a-4280593474db None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:02:58.653622 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b58401f-ddb0-4388-8c6a-4280593474db None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:02:58.653897 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b58401f-ddb0-4388-8c6a-4280593474db None admin] PATH_INFO: `/v3/users` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:02:58.654625 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b58401f-ddb0-4388-8c6a-4280593474db None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:02:58.654931 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b58401f-ddb0-4388-8c6a-4280593474db None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:02:58.656392 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b58401f-ddb0-4388-8c6a-4280593474db None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:02:58.675757 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 636/1240] 10.222.0.55 () {56 vars in 1143 bytes} [Fri Sep 23 15:02:58 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:02:58.682664 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e3b0dd-677e-4752-906e-56311ea5a9c1 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:02:58.791701 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e3b0dd-677e-4752-906e-56311ea5a9c1 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:02:58.827727 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26aa7a9f-4115-4677-acd1-a3d29d494fab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:02:58.828471 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26aa7a9f-4115-4677-acd1-a3d29d494fab None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:02:58.828699 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26aa7a9f-4115-4677-acd1-a3d29d494fab None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:02:58.828973 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26aa7a9f-4115-4677-acd1-a3d29d494fab None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:02:58.896589 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:02:58.904142 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26aa7a9f-4115-4677-acd1-a3d29d494fab None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:02:58.905613 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26aa7a9f-4115-4677-acd1-a3d29d494fab None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:02:59.020409 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 605/1241] 10.222.0.55 () {58 vars in 1282 bytes} [Fri Sep 23 15:02:58 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 23 15:02:59.027353 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b58401f-ddb0-4388-8c6a-4280593474db None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:02:59.057767 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:02:59.134782 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b58401f-ddb0-4388-8c6a-4280593474db None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:02:59.170889 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc08c0d-12dc-477b-affa-5f06266c8327 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:02:59.171437 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc08c0d-12dc-477b-affa-5f06266c8327 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:02:59.171718 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc08c0d-12dc-477b-affa-5f06266c8327 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:02:59.172025 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc08c0d-12dc-477b-affa-5f06266c8327 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:02:59.182873 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc08c0d-12dc-477b-affa-5f06266c8327 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:02:59.184356 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc08c0d-12dc-477b-affa-5f06266c8327 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:02:59.199729 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-8bc08c0d-12dc-477b-affa-5f06266c8327 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 15:02:59.199729 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:02:59.199729 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:02:59.199729 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:02:59.199729 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:02:59.199729 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:02:59.199729 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:02:59.199729 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:02:59.199729 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:02:59.199729 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:02:59.199729 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:02:59.199729 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:02:59.199729 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 15:02:59.199729 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 15:02:59.199729 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 15:02:59.199729 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 15:02:59.201145 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:02:59.201145 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:02:59.201145 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:02:59.201145 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:02:59.201145 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:02:59.201145 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:02:59.201145 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:02:59.201145 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Sep 23 15:02:59.201145 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:02:59.201145 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:02:59.201145 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:02:59.201145 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:02:59.201145 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:02:59.201145 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:02:59.201145 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:02:59.201145 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:02:59.201145 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 15:02:59.204829 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 15:02:59.204829 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 15:02:59.204829 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Sep 23 15:02:59.204829 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 637/1242] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Sep 23 15:02:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:02:59.207246 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26aa7a9f-4115-4677-acd1-a3d29d494fab None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:02:59.311606 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:02:59.317518 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26aa7a9f-4115-4677-acd1-a3d29d494fab None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:02:59.361357 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471b65f6-6854-4f97-a599-778e55b380d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:02:59.362016 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471b65f6-6854-4f97-a599-778e55b380d1 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:02:59.362236 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471b65f6-6854-4f97-a599-778e55b380d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:02:59.362503 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471b65f6-6854-4f97-a599-778e55b380d1 None admin] PATH_INFO: `/v3/domains` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:02:59.363137 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471b65f6-6854-4f97-a599-778e55b380d1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:02:59.363433 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471b65f6-6854-4f97-a599-778e55b380d1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:02:59.364830 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471b65f6-6854-4f97-a599-778e55b380d1 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:02:59.377328 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 606/1243] 10.222.0.55 () {56 vars in 1109 bytes} [Fri Sep 23 15:02:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:02:59.383762 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc08c0d-12dc-477b-affa-5f06266c8327 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:02:59.470721 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:02:59.493201 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc08c0d-12dc-477b-affa-5f06266c8327 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:02:59.527889 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564bb9c5-f0a4-4af2-9675-f72d4259b858 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:02:59.528564 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564bb9c5-f0a4-4af2-9675-f72d4259b858 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:02:59.528849 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564bb9c5-f0a4-4af2-9675-f72d4259b858 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:02:59.529131 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564bb9c5-f0a4-4af2-9675-f72d4259b858 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:02:59.536364 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564bb9c5-f0a4-4af2-9675-f72d4259b858 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:02:59.538391 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564bb9c5-f0a4-4af2-9675-f72d4259b858 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:02:59.556839 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-564bb9c5-f0a4-4af2-9675-f72d4259b858 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 15:02:59.556839 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:02:59.556839 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:02:59.556839 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:02:59.556839 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:02:59.556839 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:02:59.556839 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:02:59.556839 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:02:59.556839 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:02:59.556839 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:02:59.556839 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:02:59.556839 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:02:59.556839 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 23 15:02:59.556839 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 23 15:02:59.556839 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 23 15:02:59.558279 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 23 15:02:59.558279 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:02:59.558279 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:02:59.558279 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:02:59.558279 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:02:59.558279 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:02:59.558279 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:02:59.558279 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:02:59.558279 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Sep 23 15:02:59.558279 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:02:59.558279 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:02:59.558279 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:02:59.558279 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:02:59.558279 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:02:59.558279 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:02:59.558279 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:02:59.558279 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:02:59.559219 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 23 15:02:59.559219 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 23 15:02:59.559219 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 23 15:02:59.559219 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 23 15:02:59.559219 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 23 15:02:59.559219 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 23 15:02:59.559219 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 15:02:59.559219 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Sep 23 15:02:59.559219 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 638/1244] 10.222.0.55 () {56 vars in 1111 bytes} [Fri Sep 23 15:02:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:02:59.564052 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471b65f6-6854-4f97-a599-778e55b380d1 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:02:59.669793 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471b65f6-6854-4f97-a599-778e55b380d1 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:02:59.706805 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e08e20-1f31-4cfc-ac4d-bfe021ac3cee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:02:59.707414 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e08e20-1f31-4cfc-ac4d-bfe021ac3cee None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:02:59.707694 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e08e20-1f31-4cfc-ac4d-bfe021ac3cee None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:02:59.707964 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e08e20-1f31-4cfc-ac4d-bfe021ac3cee None admin] PATH_INFO: `/v3/projects` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:02:59.708689 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e08e20-1f31-4cfc-ac4d-bfe021ac3cee None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:02:59.708994 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e08e20-1f31-4cfc-ac4d-bfe021ac3cee None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:02:59.710646 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e08e20-1f31-4cfc-ac4d-bfe021ac3cee None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:02:59.722919 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 607/1245] 10.222.0.55 () {56 vars in 1148 bytes} [Fri Sep 23 15:02:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:02:59.730320 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564bb9c5-f0a4-4af2-9675-f72d4259b858 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:02:59.838652 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564bb9c5-f0a4-4af2-9675-f72d4259b858 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:02:59.873866 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47151e42-0274-48ba-88ef-f9182bc92bcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:02:59.874478 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47151e42-0274-48ba-88ef-f9182bc92bcc None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:02:59.874761 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47151e42-0274-48ba-88ef-f9182bc92bcc None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:02:59.875031 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47151e42-0274-48ba-88ef-f9182bc92bcc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:02:59.875751 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47151e42-0274-48ba-88ef-f9182bc92bcc None admin] RBAC: Adding query filter params (role.id=0665869b71fa40b0b552efdd57faf023, scope.project.id=87b3e9d4b6474bd4a04ba59ef52bf3e2, user.id=09f1a55d180642c781a6e0b017d9f0aa) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:02:59.876084 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47151e42-0274-48ba-88ef-f9182bc92bcc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:02:59.877512 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47151e42-0274-48ba-88ef-f9182bc92bcc None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:02:59.883415 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 639/1246] 10.222.0.55 () {56 vars in 1374 bytes} [Fri Sep 23 15:02:59 2022] GET /identity/v3/role_assignments?user.id=09f1a55d180642c781a6e0b017d9f0aa&scope.project.id=87b3e9d4b6474bd4a04ba59ef52bf3e2&role.id=0665869b71fa40b0b552efdd57faf023 => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:03:02.225874 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08b2eed-fdd9-4a9e-9c0e-50d03bdc5f86 None None] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:03:02.226430 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08b2eed-fdd9-4a9e-9c0e-50d03bdc5f86 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:03:02.226544 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08b2eed-fdd9-4a9e-9c0e-50d03bdc5f86 None None] PATH_INFO: `/` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:03:02.227861 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 608/1247] 10.222.0.55 () {54 vars in 927 bytes} [Fri Sep 23 15:03:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:03:02.233477 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abca53b1-1d0c-4393-b653-4a24f366a10d None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:03:02.241541 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abca53b1-1d0c-4393-b653-4a24f366a10d None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:03:02.241541 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abca53b1-1d0c-4393-b653-4a24f366a10d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:03:02.241737 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:03:02.260349 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-abca53b1-1d0c-4393-b653-4a24f366a10d None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:03:02.261593 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-abca53b1-1d0c-4393-b653-4a24f366a10d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:03:02.337497 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 640/1248] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Sep 23 15:03:02 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 23 15:03:02.608394 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08b2eed-fdd9-4a9e-9c0e-50d03bdc5f86 None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:03:02.632814 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:03:02.723882 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08b2eed-fdd9-4a9e-9c0e-50d03bdc5f86 None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:03:02.759513 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836a6c89-ac1b-4921-923c-f6a259625191 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:03:02.760177 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836a6c89-ac1b-4921-923c-f6a259625191 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:03:02.760455 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836a6c89-ac1b-4921-923c-f6a259625191 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:03:02.760725 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836a6c89-ac1b-4921-923c-f6a259625191 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:03:02.770861 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836a6c89-ac1b-4921-923c-f6a259625191 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 15:03:02.776497 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application [None req-836a6c89-ac1b-4921-923c-f6a259625191 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 23 15:03:02.776497 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:03:02.776497 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:03:02.776497 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:03:02.776497 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:03:02.776497 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:03:02.776497 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:03:02.776497 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:03:02.776497 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:03:02.776497 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:03:02.776497 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:03:02.776497 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:03:02.776497 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 15:03:02.776497 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 15:03:02.776497 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 15:03:02.776497 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 15:03:02.777782 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 15:03:02.777782 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 15:03:02.777782 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 15:03:02.777782 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 15:03:02.777782 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:03:02.777782 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:03:02.777782 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:03:02.777782 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:03:02.777782 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:03:02.777782 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:03:02.777782 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:03:02.777782 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application with Lock( Sep 23 15:03:02.777782 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:03:02.777782 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:03:02.777782 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:03:02.777782 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:03:02.779029 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:03:02.779029 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:03:02.779029 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:03:02.779029 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:03:02.779029 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 15:03:02.779029 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 15:03:02.779029 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 15:03:02.779029 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 15:03:02.779029 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 15:03:02.779029 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 15:03:02.779029 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 23 15:03:02.779029 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Sep 23 15:03:02.779029 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 609/1249] 10.222.0.55 () {56 vars in 1096 bytes} [Fri Sep 23 15:03:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:03:02.783926 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abca53b1-1d0c-4393-b653-4a24f366a10d None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:03:02.888466 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abca53b1-1d0c-4393-b653-4a24f366a10d None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:03:02.919157 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:03:02.924261 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cff186-1fcc-4714-8370-8e3f4420961f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:03:02.924884 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cff186-1fcc-4714-8370-8e3f4420961f None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:03:02.925169 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cff186-1fcc-4714-8370-8e3f4420961f None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:03:02.925446 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cff186-1fcc-4714-8370-8e3f4420961f None admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:03:02.926149 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cff186-1fcc-4714-8370-8e3f4420961f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:03:02.926457 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cff186-1fcc-4714-8370-8e3f4420961f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:03:02.928516 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cff186-1fcc-4714-8370-8e3f4420961f None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:03:02.937168 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 641/1250] 10.222.0.55 () {56 vars in 1099 bytes} [Fri Sep 23 15:03:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:03:02.943146 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836a6c89-ac1b-4921-923c-f6a259625191 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:03:03.034415 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:03:03.060577 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836a6c89-ac1b-4921-923c-f6a259625191 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:03:03.101865 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03c20aa-099e-48ab-86d2-9f3f8b4aeb72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:03:03.102506 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03c20aa-099e-48ab-86d2-9f3f8b4aeb72 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:03:03.102786 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03c20aa-099e-48ab-86d2-9f3f8b4aeb72 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:03:03.103058 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03c20aa-099e-48ab-86d2-9f3f8b4aeb72 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:03:03.114341 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03c20aa-099e-48ab-86d2-9f3f8b4aeb72 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:03:03.116466 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03c20aa-099e-48ab-86d2-9f3f8b4aeb72 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:03:03.130506 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application [None req-f03c20aa-099e-48ab-86d2-9f3f8b4aeb72 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 15:03:03.130506 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:03:03.130506 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:03:03.130506 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:03:03.130506 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:03:03.130506 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:03:03.130506 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:03:03.130506 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:03:03.130506 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:03:03.130506 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:03:03.130506 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:03:03.130506 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:03:03.130506 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 15:03:03.130506 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 15:03:03.130506 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 15:03:03.130506 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 15:03:03.132338 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:03:03.132338 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:03:03.132338 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:03:03.132338 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:03:03.132338 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:03:03.132338 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:03:03.132338 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:03:03.132338 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application with Lock( Sep 23 15:03:03.132338 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:03:03.132338 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:03:03.132338 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:03:03.132338 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:03:03.132338 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:03:03.132338 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:03:03.132338 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:03:03.132338 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:03:03.132338 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 15:03:03.133399 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 15:03:03.133399 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 15:03:03.133399 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Sep 23 15:03:03.133399 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 610/1251] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Sep 23 15:03:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:03:03.138176 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cff186-1fcc-4714-8370-8e3f4420961f None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:03:03.241161 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cff186-1fcc-4714-8370-8e3f4420961f None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:03:03.276496 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89827d4-17c6-4780-8842-30e41d753ddc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:03:03.277104 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89827d4-17c6-4780-8842-30e41d753ddc None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:03:03.277475 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89827d4-17c6-4780-8842-30e41d753ddc None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:03:03.277752 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89827d4-17c6-4780-8842-30e41d753ddc None admin] PATH_INFO: `/v3/domains` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:03:03.278386 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89827d4-17c6-4780-8842-30e41d753ddc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:03:03.278698 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89827d4-17c6-4780-8842-30e41d753ddc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:03:03.280114 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89827d4-17c6-4780-8842-30e41d753ddc None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:03:03.292985 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 642/1252] 10.222.0.55 () {56 vars in 1109 bytes} [Fri Sep 23 15:03:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:03:03.298821 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03c20aa-099e-48ab-86d2-9f3f8b4aeb72 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:03:03.402974 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03c20aa-099e-48ab-86d2-9f3f8b4aeb72 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:03:03.428127 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:03:03.435501 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52613f09-1839-4b5c-abff-c849bb7744d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:03:03.436157 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52613f09-1839-4b5c-abff-c849bb7744d4 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:03:03.436434 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52613f09-1839-4b5c-abff-c849bb7744d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:03:03.436702 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52613f09-1839-4b5c-abff-c849bb7744d4 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:03:03.444563 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52613f09-1839-4b5c-abff-c849bb7744d4 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:03:03.446030 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52613f09-1839-4b5c-abff-c849bb7744d4 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:03:03.456715 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application [None req-52613f09-1839-4b5c-abff-c849bb7744d4 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 23 15:03:03.456715 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:03:03.456715 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:03:03.456715 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:03:03.456715 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:03:03.456715 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:03:03.456715 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:03:03.456715 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:03:03.456715 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:03:03.456715 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:03:03.456715 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:03:03.456715 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:03:03.456715 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 23 15:03:03.456715 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 23 15:03:03.456715 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 23 15:03:03.456715 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 23 15:03:03.457898 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:03:03.457898 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:03:03.457898 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 23 15:03:03.457898 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 15:03:03.457898 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 23 15:03:03.457898 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 15:03:03.457898 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:03:03.457898 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:03:03.457898 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:03:03.457898 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:03:03.457898 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:03:03.457898 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application with Lock( Sep 23 15:03:03.457898 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:03:03.457898 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:03:03.457898 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:03:03.457898 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:03:03.457898 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:03:03.457898 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:03:03.460591 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:03:03.460591 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:03:03.460591 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 23 15:03:03.460591 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 23 15:03:03.460591 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 23 15:03:03.460591 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 23 15:03:03.460591 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 23 15:03:03.460591 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 23 15:03:03.460591 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 23 15:03:03.460591 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Sep 23 15:03:03.460591 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 611/1253] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Sep 23 15:03:03 2022] GET /identity/v3/users/placement => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:03:03.464353 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89827d4-17c6-4780-8842-30e41d753ddc None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:03:03.524386 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:03:03.568954 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89827d4-17c6-4780-8842-30e41d753ddc None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:03:03.609275 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3a47c4-c5f2-4e39-8114-350fef43a1ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:03:03.609922 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3a47c4-c5f2-4e39-8114-350fef43a1ac None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:03:03.610203 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3a47c4-c5f2-4e39-8114-350fef43a1ac None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:03:03.610479 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3a47c4-c5f2-4e39-8114-350fef43a1ac None admin] PATH_INFO: `/v3/users` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:03:03.611201 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3a47c4-c5f2-4e39-8114-350fef43a1ac None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:03:03.611508 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3a47c4-c5f2-4e39-8114-350fef43a1ac None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:03:03.612996 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3a47c4-c5f2-4e39-8114-350fef43a1ac None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:03:03.630531 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 643/1254] 10.222.0.55 () {56 vars in 1143 bytes} [Fri Sep 23 15:03:03 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:03:03.636709 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52613f09-1839-4b5c-abff-c849bb7744d4 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:03:03.743352 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52613f09-1839-4b5c-abff-c849bb7744d4 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:03:03.779269 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26afdde7-641c-431a-baf9-3e0ec012bfe1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:03:03.779800 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26afdde7-641c-431a-baf9-3e0ec012bfe1 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:03:03.780112 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26afdde7-641c-431a-baf9-3e0ec012bfe1 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:03:03.780392 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26afdde7-641c-431a-baf9-3e0ec012bfe1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:03:03.840210 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:03:03.853429 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26afdde7-641c-431a-baf9-3e0ec012bfe1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:03:03.854938 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26afdde7-641c-431a-baf9-3e0ec012bfe1 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:03:03.952499 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 612/1255] 10.222.0.55 () {58 vars in 1282 bytes} [Fri Sep 23 15:03:03 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 23 15:03:03.958455 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3a47c4-c5f2-4e39-8114-350fef43a1ac None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:03:04.069701 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3a47c4-c5f2-4e39-8114-350fef43a1ac None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:03:04.083168 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:03:04.104266 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da91669-0b2f-4c1c-bcc9-f11ab60795b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:03:04.104898 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da91669-0b2f-4c1c-bcc9-f11ab60795b1 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:03:04.112980 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da91669-0b2f-4c1c-bcc9-f11ab60795b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:03:04.112980 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da91669-0b2f-4c1c-bcc9-f11ab60795b1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:03:04.115785 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da91669-0b2f-4c1c-bcc9-f11ab60795b1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:03:04.117252 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da91669-0b2f-4c1c-bcc9-f11ab60795b1 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:03:04.132483 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-6da91669-0b2f-4c1c-bcc9-f11ab60795b1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 15:03:04.132483 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:03:04.132483 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:03:04.132483 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:03:04.132483 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:03:04.132483 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:03:04.132483 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:03:04.132483 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:03:04.132483 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:03:04.132483 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:03:04.132483 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:03:04.132483 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:03:04.132483 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 15:03:04.132483 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 15:03:04.132483 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 15:03:04.132483 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 15:03:04.136652 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:03:04.136652 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:03:04.136652 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:03:04.136652 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:03:04.136652 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:03:04.136652 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:03:04.136652 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:03:04.136652 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Sep 23 15:03:04.136652 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:03:04.136652 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:03:04.136652 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:03:04.136652 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:03:04.136652 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:03:04.136652 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:03:04.136652 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:03:04.136652 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:03:04.136652 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 15:03:04.137762 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 15:03:04.137762 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 15:03:04.137762 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Sep 23 15:03:04.137762 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 644/1256] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Sep 23 15:03:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:03:04.139658 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26afdde7-641c-431a-baf9-3e0ec012bfe1 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:03:04.226906 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:03:04.238647 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26afdde7-641c-431a-baf9-3e0ec012bfe1 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:03:04.272828 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3fdc25-7a37-443c-8b99-40f43d830437 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:03:04.273429 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3fdc25-7a37-443c-8b99-40f43d830437 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:03:04.273807 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3fdc25-7a37-443c-8b99-40f43d830437 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:03:04.274080 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3fdc25-7a37-443c-8b99-40f43d830437 None admin] PATH_INFO: `/v3/domains` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:03:04.274711 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3fdc25-7a37-443c-8b99-40f43d830437 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:03:04.275010 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3fdc25-7a37-443c-8b99-40f43d830437 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:03:04.276395 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3fdc25-7a37-443c-8b99-40f43d830437 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:03:04.290272 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 613/1257] 10.222.0.55 () {56 vars in 1109 bytes} [Fri Sep 23 15:03:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:03:04.296057 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da91669-0b2f-4c1c-bcc9-f11ab60795b1 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:03:04.401628 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da91669-0b2f-4c1c-bcc9-f11ab60795b1 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:03:04.434436 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930bb1d6-e006-4f4f-a562-b802acd161ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:03:04.435050 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930bb1d6-e006-4f4f-a562-b802acd161ce None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:03:04.435328 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930bb1d6-e006-4f4f-a562-b802acd161ce None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:03:04.435599 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930bb1d6-e006-4f4f-a562-b802acd161ce None admin] PATH_INFO: `/v3/projects/service` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:03:04.442278 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930bb1d6-e006-4f4f-a562-b802acd161ce None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:03:04.443784 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930bb1d6-e006-4f4f-a562-b802acd161ce None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:03:04.454792 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-930bb1d6-e006-4f4f-a562-b802acd161ce None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 15:03:04.454792 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:03:04.454792 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:03:04.454792 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:03:04.454792 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:03:04.454792 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:03:04.454792 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:03:04.454792 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:03:04.454792 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:03:04.454792 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:03:04.454792 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:03:04.454792 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:03:04.454792 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 23 15:03:04.454792 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 23 15:03:04.454792 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 23 15:03:04.459436 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 23 15:03:04.459436 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:03:04.459436 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:03:04.459436 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:03:04.459436 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:03:04.459436 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:03:04.459436 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:03:04.459436 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:03:04.459436 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Sep 23 15:03:04.459436 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:03:04.459436 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:03:04.459436 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:03:04.459436 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:03:04.459436 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:03:04.459436 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:03:04.459436 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:03:04.459436 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:03:04.460452 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 23 15:03:04.460452 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 23 15:03:04.460452 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 23 15:03:04.460452 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 23 15:03:04.460452 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 23 15:03:04.460452 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 23 15:03:04.460452 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 15:03:04.460452 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Sep 23 15:03:04.460452 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 645/1258] 10.222.0.55 () {56 vars in 1111 bytes} [Fri Sep 23 15:03:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 23 15:03:04.462764 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3fdc25-7a37-443c-8b99-40f43d830437 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:03:04.563663 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3fdc25-7a37-443c-8b99-40f43d830437 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:03:04.597260 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac929e53-0e04-4c5f-ade3-52087837e6a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:03:04.597857 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac929e53-0e04-4c5f-ade3-52087837e6a2 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:03:04.598134 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac929e53-0e04-4c5f-ade3-52087837e6a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:03:04.598399 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac929e53-0e04-4c5f-ade3-52087837e6a2 None admin] PATH_INFO: `/v3/projects` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:03:04.599072 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac929e53-0e04-4c5f-ade3-52087837e6a2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:03:04.599396 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac929e53-0e04-4c5f-ade3-52087837e6a2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:03:04.601790 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac929e53-0e04-4c5f-ade3-52087837e6a2 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:03:04.613656 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 614/1259] 10.222.0.55 () {56 vars in 1148 bytes} [Fri Sep 23 15:03:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:03:04.619999 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930bb1d6-e006-4f4f-a562-b802acd161ce None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:03:04.645804 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:03:04.722062 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930bb1d6-e006-4f4f-a562-b802acd161ce None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:03:04.757002 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649f4b5d-7f0a-413d-922a-dc4c20c75496 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:03:04.757611 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649f4b5d-7f0a-413d-922a-dc4c20c75496 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:03:04.757906 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649f4b5d-7f0a-413d-922a-dc4c20c75496 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:03:04.758203 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649f4b5d-7f0a-413d-922a-dc4c20c75496 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:03:04.758904 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649f4b5d-7f0a-413d-922a-dc4c20c75496 None admin] RBAC: Adding query filter params (role.id=8e2919e79df040e8926f21607818b58c, scope.project.id=87b3e9d4b6474bd4a04ba59ef52bf3e2, user.id=09f1a55d180642c781a6e0b017d9f0aa) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:03:04.759212 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649f4b5d-7f0a-413d-922a-dc4c20c75496 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:03:04.760705 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649f4b5d-7f0a-413d-922a-dc4c20c75496 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:03:04.766051 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 646/1260] 10.222.0.55 () {56 vars in 1374 bytes} [Fri Sep 23 15:03:04 2022] GET /identity/v3/role_assignments?user.id=09f1a55d180642c781a6e0b017d9f0aa&scope.project.id=87b3e9d4b6474bd4a04ba59ef52bf3e2&role.id=8e2919e79df040e8926f21607818b58c => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:03:06.916261 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e1dd6f-7f4e-400f-a9d9-c2de4201fe69 None None] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:03:06.916796 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e1dd6f-7f4e-400f-a9d9-c2de4201fe69 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:03:06.916796 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e1dd6f-7f4e-400f-a9d9-c2de4201fe69 None None] PATH_INFO: `/` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:03:06.918342 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 615/1261] 10.222.0.55 () {54 vars in 927 bytes} [Fri Sep 23 15:03:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:03:06.924221 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8480d9f4-8172-4b0c-9d64-e3c6397d42a0 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:03:06.924644 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8480d9f4-8172-4b0c-9d64-e3c6397d42a0 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:03:06.924785 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8480d9f4-8172-4b0c-9d64-e3c6397d42a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:03:06.948320 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-8480d9f4-8172-4b0c-9d64-e3c6397d42a0 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:03:06.950578 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-8480d9f4-8172-4b0c-9d64-e3c6397d42a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:03:07.021740 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 647/1262] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Sep 23 15:03:06 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 23 15:03:07.297001 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e1dd6f-7f4e-400f-a9d9-c2de4201fe69 None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:03:07.309226 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:03:07.407605 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e1dd6f-7f4e-400f-a9d9-c2de4201fe69 None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:03:07.447777 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd4692d-f3bb-4962-a670-6e0fe7576115 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:03:07.448446 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd4692d-f3bb-4962-a670-6e0fe7576115 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:03:07.448757 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd4692d-f3bb-4962-a670-6e0fe7576115 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:03:07.449034 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd4692d-f3bb-4962-a670-6e0fe7576115 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:03:07.460348 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd4692d-f3bb-4962-a670-6e0fe7576115 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 15:03:07.465104 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application [None req-9dd4692d-f3bb-4962-a670-6e0fe7576115 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 23 15:03:07.465104 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:03:07.465104 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:03:07.465104 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:03:07.465104 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:03:07.465104 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:03:07.465104 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:03:07.465104 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:03:07.465104 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:03:07.465104 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:03:07.465104 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:03:07.465104 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:03:07.465104 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 15:03:07.465104 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 15:03:07.465104 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 15:03:07.465104 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 15:03:07.466222 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 15:03:07.466222 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 15:03:07.466222 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 15:03:07.466222 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 15:03:07.466222 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:03:07.466222 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:03:07.466222 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:03:07.466222 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:03:07.466222 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:03:07.466222 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:03:07.466222 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:03:07.466222 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application with Lock( Sep 23 15:03:07.466222 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:03:07.466222 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:03:07.466222 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:03:07.466222 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:03:07.467077 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:03:07.467077 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:03:07.467077 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:03:07.467077 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:03:07.467077 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 15:03:07.467077 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 15:03:07.467077 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 15:03:07.467077 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 15:03:07.467077 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 15:03:07.467077 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 15:03:07.467077 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 23 15:03:07.467077 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Sep 23 15:03:07.467077 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 616/1263] 10.222.0.55 () {56 vars in 1096 bytes} [Fri Sep 23 15:03:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:03:07.473889 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8480d9f4-8172-4b0c-9d64-e3c6397d42a0 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:03:07.493554 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:03:07.582341 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8480d9f4-8172-4b0c-9d64-e3c6397d42a0 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:03:07.615594 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53777e5e-3135-471d-a1ba-344fce587317 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:03:07.616227 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53777e5e-3135-471d-a1ba-344fce587317 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:03:07.616509 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53777e5e-3135-471d-a1ba-344fce587317 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:03:07.616777 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53777e5e-3135-471d-a1ba-344fce587317 None admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:03:07.617424 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53777e5e-3135-471d-a1ba-344fce587317 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:03:07.617724 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53777e5e-3135-471d-a1ba-344fce587317 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:03:07.619067 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53777e5e-3135-471d-a1ba-344fce587317 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:03:07.627737 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 648/1264] 10.222.0.55 () {56 vars in 1099 bytes} [Fri Sep 23 15:03:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:03:07.633689 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd4692d-f3bb-4962-a670-6e0fe7576115 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:03:07.707000 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:03:07.752177 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd4692d-f3bb-4962-a670-6e0fe7576115 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:03:07.787336 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421449ff-edeb-4626-b9e4-e942685d7047 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:03:07.787889 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421449ff-edeb-4626-b9e4-e942685d7047 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:03:07.788199 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421449ff-edeb-4626-b9e4-e942685d7047 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:03:07.788475 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421449ff-edeb-4626-b9e4-e942685d7047 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:03:07.799147 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421449ff-edeb-4626-b9e4-e942685d7047 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:03:07.800736 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421449ff-edeb-4626-b9e4-e942685d7047 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:03:07.814874 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application [None req-421449ff-edeb-4626-b9e4-e942685d7047 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 15:03:07.814874 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:03:07.814874 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:03:07.814874 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:03:07.814874 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:03:07.814874 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:03:07.814874 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:03:07.814874 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:03:07.814874 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:03:07.814874 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:03:07.814874 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:03:07.814874 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:03:07.814874 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 15:03:07.814874 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 15:03:07.814874 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 15:03:07.814874 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 15:03:07.815924 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:03:07.815924 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:03:07.815924 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:03:07.815924 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:03:07.815924 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:03:07.815924 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:03:07.815924 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:03:07.815924 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application with Lock( Sep 23 15:03:07.815924 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:03:07.815924 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:03:07.815924 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:03:07.815924 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:03:07.815924 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:03:07.815924 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:03:07.815924 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:03:07.815924 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:03:07.815924 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 15:03:07.817240 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 15:03:07.817240 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 15:03:07.817240 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Sep 23 15:03:07.817240 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 617/1265] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Sep 23 15:03:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:03:07.822312 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53777e5e-3135-471d-a1ba-344fce587317 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:03:07.906206 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:03:07.924248 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53777e5e-3135-471d-a1ba-344fce587317 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:03:07.956382 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12fb5d2-1a00-4212-95f4-5dcc74bf66ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:03:07.956982 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12fb5d2-1a00-4212-95f4-5dcc74bf66ed None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:03:07.957260 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12fb5d2-1a00-4212-95f4-5dcc74bf66ed None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:03:07.957539 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12fb5d2-1a00-4212-95f4-5dcc74bf66ed None admin] PATH_INFO: `/v3/domains` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:03:07.958287 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12fb5d2-1a00-4212-95f4-5dcc74bf66ed None admin] RBAC: Adding query filter params (name=Default) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:03:07.959127 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12fb5d2-1a00-4212-95f4-5dcc74bf66ed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:03:07.960580 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12fb5d2-1a00-4212-95f4-5dcc74bf66ed None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:03:07.974139 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 649/1266] 10.222.0.55 () {56 vars in 1109 bytes} [Fri Sep 23 15:03:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:03:07.980244 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421449ff-edeb-4626-b9e4-e942685d7047 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:03:08.084286 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421449ff-edeb-4626-b9e4-e942685d7047 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:03:08.105077 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:03:08.118666 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff33f47b-00da-4765-89bb-96b59e8d4f63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:03:08.119296 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff33f47b-00da-4765-89bb-96b59e8d4f63 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:03:08.119578 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff33f47b-00da-4765-89bb-96b59e8d4f63 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:03:08.119935 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff33f47b-00da-4765-89bb-96b59e8d4f63 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:03:08.127724 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff33f47b-00da-4765-89bb-96b59e8d4f63 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:03:08.129219 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff33f47b-00da-4765-89bb-96b59e8d4f63 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:03:08.140366 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application [None req-ff33f47b-00da-4765-89bb-96b59e8d4f63 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 23 15:03:08.140366 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:03:08.140366 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:03:08.140366 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:03:08.140366 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:03:08.140366 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:03:08.140366 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:03:08.140366 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:03:08.140366 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:03:08.140366 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:03:08.140366 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:03:08.140366 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:03:08.140366 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 23 15:03:08.140366 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 23 15:03:08.140366 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 23 15:03:08.140366 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 23 15:03:08.141565 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:03:08.141565 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:03:08.141565 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 23 15:03:08.141565 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 15:03:08.141565 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 23 15:03:08.141565 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 15:03:08.141565 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:03:08.141565 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:03:08.141565 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:03:08.141565 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:03:08.141565 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:03:08.141565 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application with Lock( Sep 23 15:03:08.141565 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:03:08.141565 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:03:08.141565 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:03:08.141565 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:03:08.141565 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:03:08.141565 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:03:08.142519 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:03:08.142519 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:03:08.142519 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 23 15:03:08.142519 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 23 15:03:08.142519 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 23 15:03:08.142519 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 23 15:03:08.142519 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 23 15:03:08.142519 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 23 15:03:08.142519 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 23 15:03:08.142519 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Sep 23 15:03:08.142519 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 618/1267] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Sep 23 15:03:07 2022] GET /identity/v3/users/placement => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:03:08.148114 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12fb5d2-1a00-4212-95f4-5dcc74bf66ed None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:03:08.247439 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12fb5d2-1a00-4212-95f4-5dcc74bf66ed None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:03:08.281055 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59afe2c9-03c9-4cb5-9f7c-551e85fef216 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:03:08.281660 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59afe2c9-03c9-4cb5-9f7c-551e85fef216 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:03:08.281945 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59afe2c9-03c9-4cb5-9f7c-551e85fef216 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:03:08.282223 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59afe2c9-03c9-4cb5-9f7c-551e85fef216 None admin] PATH_INFO: `/v3/users` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:03:08.282940 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59afe2c9-03c9-4cb5-9f7c-551e85fef216 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:03:08.283247 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59afe2c9-03c9-4cb5-9f7c-551e85fef216 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:03:08.284948 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59afe2c9-03c9-4cb5-9f7c-551e85fef216 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:03:08.294300 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:03:08.302907 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 650/1268] 10.222.0.55 () {56 vars in 1143 bytes} [Fri Sep 23 15:03:08 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:03:08.308874 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff33f47b-00da-4765-89bb-96b59e8d4f63 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:03:08.409871 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff33f47b-00da-4765-89bb-96b59e8d4f63 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:03:08.443865 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8af573-6ac3-4be7-ba33-f16f5ed5ccec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:03:08.444554 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8af573-6ac3-4be7-ba33-f16f5ed5ccec None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:03:08.449091 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8af573-6ac3-4be7-ba33-f16f5ed5ccec None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:03:08.449091 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8af573-6ac3-4be7-ba33-f16f5ed5ccec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:03:08.455819 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8af573-6ac3-4be7-ba33-f16f5ed5ccec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:03:08.457350 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8af573-6ac3-4be7-ba33-f16f5ed5ccec None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:03:08.472085 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application [None req-3c8af573-6ac3-4be7-ba33-f16f5ed5ccec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 15:03:08.472085 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:03:08.472085 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:03:08.472085 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:03:08.472085 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:03:08.472085 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:03:08.472085 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:03:08.472085 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:03:08.472085 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:03:08.472085 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:03:08.472085 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:03:08.472085 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:03:08.472085 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 15:03:08.472085 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 15:03:08.472085 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 15:03:08.472085 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 15:03:08.473243 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:03:08.473243 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:03:08.473243 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:03:08.473243 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:03:08.473243 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:03:08.473243 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:03:08.473243 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:03:08.473243 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application with Lock( Sep 23 15:03:08.473243 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:03:08.473243 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:03:08.473243 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:03:08.473243 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:03:08.473243 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:03:08.473243 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:03:08.473243 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:03:08.473243 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:03:08.473243 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 15:03:08.474173 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 15:03:08.474173 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 15:03:08.474173 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Sep 23 15:03:08.474173 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 619/1269] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Sep 23 15:03:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:03:08.479080 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59afe2c9-03c9-4cb5-9f7c-551e85fef216 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:03:08.577072 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59afe2c9-03c9-4cb5-9f7c-551e85fef216 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:03:08.610760 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df55158-60e7-42ed-9dde-d6111ca1b3c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:03:08.611407 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df55158-60e7-42ed-9dde-d6111ca1b3c2 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:03:08.611763 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df55158-60e7-42ed-9dde-d6111ca1b3c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:03:08.611965 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df55158-60e7-42ed-9dde-d6111ca1b3c2 None admin] PATH_INFO: `/v3/domains` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:03:08.612626 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df55158-60e7-42ed-9dde-d6111ca1b3c2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:03:08.612928 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df55158-60e7-42ed-9dde-d6111ca1b3c2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:03:08.614286 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df55158-60e7-42ed-9dde-d6111ca1b3c2 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:03:08.626571 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 651/1270] 10.222.0.55 () {56 vars in 1109 bytes} [Fri Sep 23 15:03:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:03:08.632757 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8af573-6ac3-4be7-ba33-f16f5ed5ccec None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:03:08.664568 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:03:08.733918 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8af573-6ac3-4be7-ba33-f16f5ed5ccec None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:03:08.767512 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb0f7f5-9eaf-4bc7-b7eb-32c1fea5afd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:03:08.768169 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb0f7f5-9eaf-4bc7-b7eb-32c1fea5afd5 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:03:08.768456 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb0f7f5-9eaf-4bc7-b7eb-32c1fea5afd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:03:08.768730 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb0f7f5-9eaf-4bc7-b7eb-32c1fea5afd5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:03:08.775518 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb0f7f5-9eaf-4bc7-b7eb-32c1fea5afd5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:03:08.777334 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb0f7f5-9eaf-4bc7-b7eb-32c1fea5afd5 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:03:08.789354 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application [None req-0fb0f7f5-9eaf-4bc7-b7eb-32c1fea5afd5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 15:03:08.789354 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:03:08.789354 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:03:08.789354 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:03:08.789354 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:03:08.789354 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:03:08.789354 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:03:08.789354 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:03:08.789354 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:03:08.789354 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:03:08.789354 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:03:08.789354 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:03:08.789354 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 23 15:03:08.789354 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 23 15:03:08.789354 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 23 15:03:08.790436 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 23 15:03:08.790436 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:03:08.790436 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:03:08.790436 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:03:08.790436 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:03:08.790436 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:03:08.790436 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:03:08.790436 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:03:08.790436 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application with Lock( Sep 23 15:03:08.790436 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:03:08.790436 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:03:08.790436 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:03:08.790436 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:03:08.790436 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:03:08.790436 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:03:08.790436 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:03:08.790436 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:03:08.791769 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 23 15:03:08.791769 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 23 15:03:08.791769 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 23 15:03:08.791769 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 23 15:03:08.791769 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 23 15:03:08.791769 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 23 15:03:08.791769 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 15:03:08.791769 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Sep 23 15:03:08.791769 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 620/1271] 10.222.0.55 () {56 vars in 1111 bytes} [Fri Sep 23 15:03:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:03:08.796540 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df55158-60e7-42ed-9dde-d6111ca1b3c2 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:03:08.858513 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:03:08.900523 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df55158-60e7-42ed-9dde-d6111ca1b3c2 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:03:08.935240 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370d330c-f74d-4699-88a3-da437ede85fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:03:08.935915 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370d330c-f74d-4699-88a3-da437ede85fb None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:03:08.936142 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370d330c-f74d-4699-88a3-da437ede85fb None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:03:08.936416 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370d330c-f74d-4699-88a3-da437ede85fb None admin] PATH_INFO: `/v3/projects` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:03:08.937150 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370d330c-f74d-4699-88a3-da437ede85fb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:03:08.937453 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370d330c-f74d-4699-88a3-da437ede85fb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:03:08.938867 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370d330c-f74d-4699-88a3-da437ede85fb None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:03:08.950832 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 652/1272] 10.222.0.55 () {56 vars in 1148 bytes} [Fri Sep 23 15:03:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:03:08.956916 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb0f7f5-9eaf-4bc7-b7eb-32c1fea5afd5 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:03:09.050634 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:03:09.060132 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb0f7f5-9eaf-4bc7-b7eb-32c1fea5afd5 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:03:09.094972 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8f27c1-1262-46e1-91a0-d149ab0875ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:03:09.095627 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8f27c1-1262-46e1-91a0-d149ab0875ed None admin] REQUEST_METHOD: `PUT` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:03:09.095903 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8f27c1-1262-46e1-91a0-d149ab0875ed None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:03:09.096200 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8f27c1-1262-46e1-91a0-d149ab0875ed None admin] PATH_INFO: `/v3/projects/87b3e9d4b6474bd4a04ba59ef52bf3e2/users/09f1a55d180642c781a6e0b017d9f0aa/roles/8e2919e79df040e8926f21607818b58c` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:03:09.098188 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8f27c1-1262-46e1-91a0-d149ab0875ed None admin] RBAC: Authorizing `identity:create_grant(project_id=87b3e9d4b6474bd4a04ba59ef52bf3e2, user_id=09f1a55d180642c781a6e0b017d9f0aa, role_id=8e2919e79df040e8926f21607818b58c)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:03:09.099812 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8f27c1-1262-46e1-91a0-d149ab0875ed None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:03:09.206596 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 621/1273] 10.222.0.55 () {58 vars in 1439 bytes} [Fri Sep 23 15:03:08 2022] PUT /identity/v3/projects/87b3e9d4b6474bd4a04ba59ef52bf3e2/users/09f1a55d180642c781a6e0b017d9f0aa/roles/8e2919e79df040e8926f21607818b58c => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:03:11.365522 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f768c6f2-73e2-4e83-97f4-b95cc6eb8e27 None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:03:11.366044 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f768c6f2-73e2-4e83-97f4-b95cc6eb8e27 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:03:11.366044 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f768c6f2-73e2-4e83-97f4-b95cc6eb8e27 None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:03:11.367310 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 653/1274] 10.222.0.55 () {54 vars in 927 bytes} [Fri Sep 23 15:03:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:03:11.372577 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d778ba26-c24a-4804-b4cc-3ce626cef830 None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:03:11.372974 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d778ba26-c24a-4804-b4cc-3ce626cef830 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:03:11.373128 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d778ba26-c24a-4804-b4cc-3ce626cef830 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:03:11.391743 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-d778ba26-c24a-4804-b4cc-3ce626cef830 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:03:11.392979 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-d778ba26-c24a-4804-b4cc-3ce626cef830 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:03:11.460814 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 622/1275] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Sep 23 15:03:11 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 23 15:03:11.731046 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f768c6f2-73e2-4e83-97f4-b95cc6eb8e27 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:03:11.838215 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f768c6f2-73e2-4e83-97f4-b95cc6eb8e27 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:03:11.857946 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:03:11.874799 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b547eafb-c7ab-4689-9451-241501f6934e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:03:11.875435 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b547eafb-c7ab-4689-9451-241501f6934e None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:03:11.875717 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b547eafb-c7ab-4689-9451-241501f6934e None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:03:11.876001 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b547eafb-c7ab-4689-9451-241501f6934e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:03:11.886529 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b547eafb-c7ab-4689-9451-241501f6934e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 15:03:11.891416 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-b547eafb-c7ab-4689-9451-241501f6934e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 23 15:03:11.891416 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:03:11.891416 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:03:11.891416 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:03:11.891416 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:03:11.891416 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:03:11.891416 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:03:11.891416 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:03:11.891416 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:03:11.891416 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:03:11.891416 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:03:11.891416 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:03:11.891416 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 15:03:11.891416 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 15:03:11.891416 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 15:03:11.891416 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 15:03:11.897577 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 15:03:11.897577 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 15:03:11.897577 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 15:03:11.897577 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 15:03:11.897577 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:03:11.897577 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:03:11.897577 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:03:11.897577 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:03:11.897577 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:03:11.897577 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:03:11.897577 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:03:11.897577 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Sep 23 15:03:11.897577 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:03:11.897577 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:03:11.897577 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:03:11.897577 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:03:11.899254 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:03:11.899254 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:03:11.899254 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:03:11.899254 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:03:11.899254 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 15:03:11.899254 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 15:03:11.899254 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 15:03:11.899254 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 15:03:11.899254 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 15:03:11.899254 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 15:03:11.899254 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 23 15:03:11.899254 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Sep 23 15:03:11.899254 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 654/1276] 10.222.0.55 () {56 vars in 1096 bytes} [Fri Sep 23 15:03:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:03:11.900727 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d778ba26-c24a-4804-b4cc-3ce626cef830 None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:03:11.984997 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:03:12.004171 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d778ba26-c24a-4804-b4cc-3ce626cef830 None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:03:12.038828 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c5102f-7198-46b2-aff2-80d7645f88a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:03:12.039709 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c5102f-7198-46b2-aff2-80d7645f88a4 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:03:12.040579 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c5102f-7198-46b2-aff2-80d7645f88a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:03:12.041232 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c5102f-7198-46b2-aff2-80d7645f88a4 None admin] PATH_INFO: `/v3/roles` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:03:12.041978 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c5102f-7198-46b2-aff2-80d7645f88a4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:03:12.042285 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c5102f-7198-46b2-aff2-80d7645f88a4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:03:12.043719 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c5102f-7198-46b2-aff2-80d7645f88a4 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:03:12.054185 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 623/1277] 10.222.0.55 () {56 vars in 1099 bytes} [Fri Sep 23 15:03:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:03:12.060258 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b547eafb-c7ab-4689-9451-241501f6934e None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:03:12.178393 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b547eafb-c7ab-4689-9451-241501f6934e None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:03:12.213330 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbbdd2d-a0af-42cf-bf1f-6bd76b683d32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:03:12.213950 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbbdd2d-a0af-42cf-bf1f-6bd76b683d32 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:03:12.214224 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbbdd2d-a0af-42cf-bf1f-6bd76b683d32 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:03:12.214492 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbbdd2d-a0af-42cf-bf1f-6bd76b683d32 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:03:12.224993 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbbdd2d-a0af-42cf-bf1f-6bd76b683d32 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:03:12.226490 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbbdd2d-a0af-42cf-bf1f-6bd76b683d32 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:03:12.241045 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-4fbbdd2d-a0af-42cf-bf1f-6bd76b683d32 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 15:03:12.241045 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:03:12.241045 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:03:12.241045 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:03:12.241045 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:03:12.241045 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:03:12.241045 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:03:12.241045 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:03:12.241045 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:03:12.241045 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:03:12.241045 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:03:12.241045 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:03:12.241045 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 15:03:12.241045 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 15:03:12.241045 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 15:03:12.241045 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 15:03:12.242174 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:03:12.242174 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:03:12.242174 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:03:12.242174 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:03:12.242174 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:03:12.242174 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:03:12.242174 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:03:12.242174 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Sep 23 15:03:12.242174 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:03:12.242174 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:03:12.242174 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:03:12.242174 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:03:12.242174 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:03:12.242174 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:03:12.242174 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:03:12.242174 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:03:12.242174 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 15:03:12.243288 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 15:03:12.243288 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 15:03:12.243288 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Sep 23 15:03:12.243288 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 655/1278] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Sep 23 15:03:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:03:12.248030 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c5102f-7198-46b2-aff2-80d7645f88a4 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:03:12.353980 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c5102f-7198-46b2-aff2-80d7645f88a4 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:03:12.388312 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d500c106-88a0-4982-ab7a-249d44be9e24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:03:12.389012 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d500c106-88a0-4982-ab7a-249d44be9e24 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:03:12.389184 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d500c106-88a0-4982-ab7a-249d44be9e24 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:03:12.389454 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d500c106-88a0-4982-ab7a-249d44be9e24 None admin] PATH_INFO: `/v3/domains` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:03:12.390086 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d500c106-88a0-4982-ab7a-249d44be9e24 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:03:12.390388 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d500c106-88a0-4982-ab7a-249d44be9e24 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:03:12.391758 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d500c106-88a0-4982-ab7a-249d44be9e24 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:03:12.404242 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 624/1279] 10.222.0.55 () {56 vars in 1109 bytes} [Fri Sep 23 15:03:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:03:12.410277 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbbdd2d-a0af-42cf-bf1f-6bd76b683d32 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:03:12.440614 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:03:12.517503 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbbdd2d-a0af-42cf-bf1f-6bd76b683d32 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:03:12.554608 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d66006-afb8-4112-9563-4ccf1f22bc6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:03:12.555236 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d66006-afb8-4112-9563-4ccf1f22bc6a None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:03:12.555521 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d66006-afb8-4112-9563-4ccf1f22bc6a None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:03:12.555800 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d66006-afb8-4112-9563-4ccf1f22bc6a None admin] PATH_INFO: `/v3/users/placement` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:03:12.563338 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d66006-afb8-4112-9563-4ccf1f22bc6a None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:03:12.564828 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d66006-afb8-4112-9563-4ccf1f22bc6a None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:03:12.578220 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-e2d66006-afb8-4112-9563-4ccf1f22bc6a None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 23 15:03:12.578220 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:03:12.578220 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:03:12.578220 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:03:12.578220 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:03:12.578220 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:03:12.578220 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:03:12.578220 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:03:12.578220 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:03:12.578220 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:03:12.578220 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:03:12.578220 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:03:12.578220 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 23 15:03:12.578220 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 23 15:03:12.578220 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 23 15:03:12.578220 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 23 15:03:12.579449 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:03:12.579449 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:03:12.579449 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 23 15:03:12.579449 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 15:03:12.579449 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 23 15:03:12.579449 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 15:03:12.579449 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:03:12.579449 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:03:12.579449 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:03:12.579449 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:03:12.579449 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:03:12.579449 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Sep 23 15:03:12.579449 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:03:12.579449 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:03:12.579449 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:03:12.579449 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:03:12.579449 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:03:12.579449 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:03:12.580798 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:03:12.580798 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:03:12.580798 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 23 15:03:12.580798 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 23 15:03:12.580798 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 23 15:03:12.580798 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 23 15:03:12.580798 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 23 15:03:12.580798 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 23 15:03:12.580798 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 23 15:03:12.580798 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Sep 23 15:03:12.584148 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 656/1280] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Sep 23 15:03:12 2022] GET /identity/v3/users/placement => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:03:12.586213 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d500c106-88a0-4982-ab7a-249d44be9e24 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:03:12.590225 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:03:12.693511 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d500c106-88a0-4982-ab7a-249d44be9e24 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:03:12.728890 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fd6db5-4e6a-400a-aaa7-bd8fc75d009c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:03:12.729500 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fd6db5-4e6a-400a-aaa7-bd8fc75d009c None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:03:12.729774 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fd6db5-4e6a-400a-aaa7-bd8fc75d009c None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:03:12.730044 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fd6db5-4e6a-400a-aaa7-bd8fc75d009c None admin] PATH_INFO: `/v3/users` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:03:12.730851 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fd6db5-4e6a-400a-aaa7-bd8fc75d009c None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:03:12.731161 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fd6db5-4e6a-400a-aaa7-bd8fc75d009c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:03:12.732609 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fd6db5-4e6a-400a-aaa7-bd8fc75d009c None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:03:12.750205 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 625/1281] 10.222.0.55 () {56 vars in 1143 bytes} [Fri Sep 23 15:03:12 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:03:12.756527 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d66006-afb8-4112-9563-4ccf1f22bc6a None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:03:12.859827 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:03:12.870447 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d66006-afb8-4112-9563-4ccf1f22bc6a None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:03:12.907299 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0919e6a-2103-49a9-b1fb-e262a33147bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:03:12.908022 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0919e6a-2103-49a9-b1fb-e262a33147bf None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:03:12.908835 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0919e6a-2103-49a9-b1fb-e262a33147bf None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:03:12.909372 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0919e6a-2103-49a9-b1fb-e262a33147bf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:03:12.988475 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0919e6a-2103-49a9-b1fb-e262a33147bf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:03:12.990285 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0919e6a-2103-49a9-b1fb-e262a33147bf None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:03:13.086011 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:03:13.101332 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 657/1282] 10.222.0.55 () {58 vars in 1282 bytes} [Fri Sep 23 15:03:12 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 23 15:03:13.107756 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fd6db5-4e6a-400a-aaa7-bd8fc75d009c None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:03:13.177508 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:03:13.215517 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fd6db5-4e6a-400a-aaa7-bd8fc75d009c None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:03:13.250038 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805ec1b6-6ee3-4b30-bffa-0d4d017bcb61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:03:13.250670 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805ec1b6-6ee3-4b30-bffa-0d4d017bcb61 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:03:13.250983 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805ec1b6-6ee3-4b30-bffa-0d4d017bcb61 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:03:13.251247 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805ec1b6-6ee3-4b30-bffa-0d4d017bcb61 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:03:13.261676 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805ec1b6-6ee3-4b30-bffa-0d4d017bcb61 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:03:13.263112 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805ec1b6-6ee3-4b30-bffa-0d4d017bcb61 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:03:13.278184 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application [None req-805ec1b6-6ee3-4b30-bffa-0d4d017bcb61 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 15:03:13.278184 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:03:13.278184 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:03:13.278184 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:03:13.278184 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:03:13.278184 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:03:13.278184 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:03:13.278184 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:03:13.278184 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:03:13.278184 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:03:13.278184 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:03:13.278184 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:03:13.278184 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 15:03:13.278184 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 15:03:13.278184 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 15:03:13.278184 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 15:03:13.279201 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:03:13.279201 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:03:13.279201 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:03:13.279201 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:03:13.279201 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:03:13.279201 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:03:13.279201 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:03:13.279201 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application with Lock( Sep 23 15:03:13.279201 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:03:13.279201 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:03:13.279201 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:03:13.279201 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:03:13.279201 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:03:13.279201 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:03:13.279201 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:03:13.279201 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:03:13.279201 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 15:03:13.285696 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 15:03:13.285696 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 15:03:13.285696 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Sep 23 15:03:13.285696 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 626/1283] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Sep 23 15:03:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:03:13.285945 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0919e6a-2103-49a9-b1fb-e262a33147bf None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:03:13.394483 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0919e6a-2103-49a9-b1fb-e262a33147bf None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:03:13.431084 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1724c3ac-f216-4f70-8854-72563647cf74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:03:13.431721 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1724c3ac-f216-4f70-8854-72563647cf74 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:03:13.431885 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1724c3ac-f216-4f70-8854-72563647cf74 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:03:13.432277 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1724c3ac-f216-4f70-8854-72563647cf74 None admin] PATH_INFO: `/v3/domains` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:03:13.432830 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1724c3ac-f216-4f70-8854-72563647cf74 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:03:13.433132 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1724c3ac-f216-4f70-8854-72563647cf74 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:03:13.434793 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1724c3ac-f216-4f70-8854-72563647cf74 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:03:13.446122 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 658/1284] 10.222.0.55 () {56 vars in 1109 bytes} [Fri Sep 23 15:03:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:03:13.452184 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805ec1b6-6ee3-4b30-bffa-0d4d017bcb61 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:03:13.556807 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805ec1b6-6ee3-4b30-bffa-0d4d017bcb61 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:03:13.571319 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:03:13.591324 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9035662b-0471-49f0-8771-1ba76795224e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:03:13.591956 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9035662b-0471-49f0-8771-1ba76795224e None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:03:13.592272 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9035662b-0471-49f0-8771-1ba76795224e None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:03:13.592542 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9035662b-0471-49f0-8771-1ba76795224e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:03:13.599785 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9035662b-0471-49f0-8771-1ba76795224e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:03:13.601244 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9035662b-0471-49f0-8771-1ba76795224e None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:03:13.612606 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application [None req-9035662b-0471-49f0-8771-1ba76795224e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 15:03:13.612606 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:03:13.612606 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:03:13.612606 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:03:13.612606 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:03:13.612606 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:03:13.612606 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:03:13.612606 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:03:13.612606 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:03:13.612606 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:03:13.612606 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:03:13.612606 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:03:13.612606 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 23 15:03:13.612606 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 23 15:03:13.612606 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 23 15:03:13.613769 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 23 15:03:13.613769 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:03:13.613769 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:03:13.613769 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:03:13.613769 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:03:13.613769 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:03:13.613769 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:03:13.613769 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:03:13.613769 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application with Lock( Sep 23 15:03:13.613769 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:03:13.613769 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:03:13.613769 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:03:13.613769 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:03:13.613769 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:03:13.613769 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:03:13.613769 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:03:13.613769 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:03:13.614971 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 23 15:03:13.614971 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 23 15:03:13.614971 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 23 15:03:13.614971 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 23 15:03:13.614971 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 23 15:03:13.614971 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 23 15:03:13.614971 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 15:03:13.614971 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: ERROR keystone.server.flask.application Sep 23 15:03:13.614971 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 627/1285] 10.222.0.55 () {56 vars in 1111 bytes} [Fri Sep 23 15:03:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:03:13.620148 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1724c3ac-f216-4f70-8854-72563647cf74 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:03:13.674387 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:03:13.725024 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1724c3ac-f216-4f70-8854-72563647cf74 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:03:13.758779 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afceb50-873a-4432-bb04-ca9b63af2101 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:03:13.759388 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afceb50-873a-4432-bb04-ca9b63af2101 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:03:13.759668 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afceb50-873a-4432-bb04-ca9b63af2101 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:03:13.759941 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afceb50-873a-4432-bb04-ca9b63af2101 None admin] PATH_INFO: `/v3/projects` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:03:13.760659 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afceb50-873a-4432-bb04-ca9b63af2101 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:03:13.760966 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afceb50-873a-4432-bb04-ca9b63af2101 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:03:13.762389 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afceb50-873a-4432-bb04-ca9b63af2101 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:03:13.775071 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 659/1286] 10.222.0.55 () {56 vars in 1148 bytes} [Fri Sep 23 15:03:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:03:13.781437 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9035662b-0471-49f0-8771-1ba76795224e None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:03:13.885020 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9035662b-0471-49f0-8771-1ba76795224e None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:03:13.921016 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb58448-1194-454d-8bc4-cf8ebba69eae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:03:13.921634 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb58448-1194-454d-8bc4-cf8ebba69eae None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:03:13.921921 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb58448-1194-454d-8bc4-cf8ebba69eae None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:03:13.922193 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb58448-1194-454d-8bc4-cf8ebba69eae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:03:13.922911 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb58448-1194-454d-8bc4-cf8ebba69eae None admin] RBAC: Adding query filter params (role.id=8e2919e79df040e8926f21607818b58c, scope.project.id=87b3e9d4b6474bd4a04ba59ef52bf3e2, user.id=09f1a55d180642c781a6e0b017d9f0aa) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:03:13.923242 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb58448-1194-454d-8bc4-cf8ebba69eae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:03:13.924770 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb58448-1194-454d-8bc4-cf8ebba69eae None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:03:13.931319 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 628/1287] 10.222.0.55 () {56 vars in 1374 bytes} [Fri Sep 23 15:03:13 2022] GET /identity/v3/role_assignments?user.id=09f1a55d180642c781a6e0b017d9f0aa&scope.project.id=87b3e9d4b6474bd4a04ba59ef52bf3e2&role.id=8e2919e79df040e8926f21607818b58c => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:03:16.150829 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83840bae-1095-4a05-b12f-8e5e7b8a6796 None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:03:16.151374 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83840bae-1095-4a05-b12f-8e5e7b8a6796 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:03:16.151374 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83840bae-1095-4a05-b12f-8e5e7b8a6796 None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:03:16.152790 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 660/1288] 10.222.0.55 () {54 vars in 927 bytes} [Fri Sep 23 15:03:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:03:16.158481 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f144eadd-952b-4a53-a0d4-f37724e8906b None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:03:16.158915 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f144eadd-952b-4a53-a0d4-f37724e8906b None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:03:16.161381 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f144eadd-952b-4a53-a0d4-f37724e8906b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:03:16.179881 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-f144eadd-952b-4a53-a0d4-f37724e8906b None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:03:16.181102 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-f144eadd-952b-4a53-a0d4-f37724e8906b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:03:16.211451 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:03:16.249831 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 629/1289] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Sep 23 15:03:16 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 23 15:03:16.525341 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83840bae-1095-4a05-b12f-8e5e7b8a6796 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:03:16.632687 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83840bae-1095-4a05-b12f-8e5e7b8a6796 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:03:16.643773 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:03:16.668453 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77cfa95-b9ea-403b-be85-f0bb1edb95e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:03:16.669083 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77cfa95-b9ea-403b-be85-f0bb1edb95e4 None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:03:16.669362 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77cfa95-b9ea-403b-be85-f0bb1edb95e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:03:16.669634 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77cfa95-b9ea-403b-be85-f0bb1edb95e4 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:03:16.670215 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77cfa95-b9ea-403b-be85-f0bb1edb95e4 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:03:16.671568 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77cfa95-b9ea-403b-be85-f0bb1edb95e4 None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:03:16.680163 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-c77cfa95-b9ea-403b-be85-f0bb1edb95e4 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 23 15:03:16.680163 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:03:16.680163 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 23 15:03:16.680163 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 23 15:03:16.680163 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 23 15:03:16.680163 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 23 15:03:16.680163 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 23 15:03:16.680163 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 23 15:03:16.680163 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 23 15:03:16.680163 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Sep 23 15:03:16.680163 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 23 15:03:16.680163 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Sep 23 15:03:16.680163 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:03:16.680163 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:03:16.680163 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:03:16.680163 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:03:16.680163 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:03:16.681481 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:03:16.681481 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:03:16.681481 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:03:16.681481 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:03:16.681481 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:03:16.681481 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:03:16.681481 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 23 15:03:16.681481 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 23 15:03:16.681481 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 23 15:03:16.681481 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 23 15:03:16.681481 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:03:16.681481 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:03:16.681481 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:03:16.681481 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:03:16.681481 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:03:16.681481 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:03:16.682638 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:03:16.682638 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Sep 23 15:03:16.682638 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:03:16.682638 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:03:16.682638 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:03:16.682638 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:03:16.682638 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:03:16.682638 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:03:16.682638 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:03:16.682638 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:03:16.682638 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 23 15:03:16.682638 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 23 15:03:16.682638 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 23 15:03:16.682638 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Sep 23 15:03:16.682638 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 661/1290] 10.222.0.55 () {56 vars in 1117 bytes} [Fri Sep 23 15:03:16 2022] GET /identity/v3/services/placement => generated 90 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:03:16.687376 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f144eadd-952b-4a53-a0d4-f37724e8906b None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:03:16.798411 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f144eadd-952b-4a53-a0d4-f37724e8906b None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:03:16.835688 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e344ee5-5ce1-4aad-a60d-99afedd8b6f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:03:16.836378 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e344ee5-5ce1-4aad-a60d-99afedd8b6f6 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:03:16.836660 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e344ee5-5ce1-4aad-a60d-99afedd8b6f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:03:16.836935 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e344ee5-5ce1-4aad-a60d-99afedd8b6f6 None admin] PATH_INFO: `/v3/services` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:03:16.837606 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e344ee5-5ce1-4aad-a60d-99afedd8b6f6 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:03:16.837909 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e344ee5-5ce1-4aad-a60d-99afedd8b6f6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:03:16.839274 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e344ee5-5ce1-4aad-a60d-99afedd8b6f6 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:03:16.844010 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 630/1291] 10.222.0.55 () {56 vars in 1116 bytes} [Fri Sep 23 15:03:16 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:03:16.849897 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77cfa95-b9ea-403b-be85-f0bb1edb95e4 None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:03:16.959759 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77cfa95-b9ea-403b-be85-f0bb1edb95e4 None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:03:16.994229 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8eea868-1f0c-49fe-8f21-d8718e33533c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:03:16.994837 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8eea868-1f0c-49fe-8f21-d8718e33533c None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:03:16.995122 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8eea868-1f0c-49fe-8f21-d8718e33533c None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:03:16.995400 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8eea868-1f0c-49fe-8f21-d8718e33533c None admin] PATH_INFO: `/v3/services` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:03:16.996055 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8eea868-1f0c-49fe-8f21-d8718e33533c None admin] RBAC: Adding query filter params (type=placement) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:03:16.996367 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8eea868-1f0c-49fe-8f21-d8718e33533c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:03:16.997731 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8eea868-1f0c-49fe-8f21-d8718e33533c None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:03:17.002131 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 662/1292] 10.222.0.55 () {56 vars in 1116 bytes} [Fri Sep 23 15:03:16 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:03:19.190011 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134aefcf-ab10-4d26-9b0d-fb8613f0cf9d None None] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:03:19.190540 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134aefcf-ab10-4d26-9b0d-fb8613f0cf9d None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:03:19.190540 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134aefcf-ab10-4d26-9b0d-fb8613f0cf9d None None] PATH_INFO: `/` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:03:19.192036 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 631/1293] 10.222.0.55 () {54 vars in 927 bytes} [Fri Sep 23 15:03:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:03:19.197402 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bd68a9-7370-46dc-b9d3-77a69bfecca2 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:03:19.197818 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bd68a9-7370-46dc-b9d3-77a69bfecca2 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:03:19.197939 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bd68a9-7370-46dc-b9d3-77a69bfecca2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:03:19.215882 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-66bd68a9-7370-46dc-b9d3-77a69bfecca2 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:03:19.217086 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-66bd68a9-7370-46dc-b9d3-77a69bfecca2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:03:19.246993 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:03:19.282334 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 663/1294] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Sep 23 15:03:19 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 23 15:03:19.550754 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134aefcf-ab10-4d26-9b0d-fb8613f0cf9d None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:03:19.582957 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:03:19.654063 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134aefcf-ab10-4d26-9b0d-fb8613f0cf9d None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:03:19.689025 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbc7405-6571-4aba-ab50-9e58203c3017 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:03:19.689627 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbc7405-6571-4aba-ab50-9e58203c3017 None admin] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:03:19.689908 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbc7405-6571-4aba-ab50-9e58203c3017 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:03:19.690181 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbc7405-6571-4aba-ab50-9e58203c3017 None admin] PATH_INFO: `/v3/services` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:03:19.690839 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbc7405-6571-4aba-ab50-9e58203c3017 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:03:19.693144 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbc7405-6571-4aba-ab50-9e58203c3017 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:03:19.790908 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 632/1295] 10.222.0.55 () {60 vars in 1141 bytes} [Fri Sep 23 15:03:19 2022] POST /identity/v3/services => generated 246 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:03:21.948978 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efd38ce-8be5-4082-bc0f-03ba4607aa97 None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:03:21.949531 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efd38ce-8be5-4082-bc0f-03ba4607aa97 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:03:21.949647 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efd38ce-8be5-4082-bc0f-03ba4607aa97 None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:03:21.950884 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 664/1296] 10.222.0.55 () {54 vars in 927 bytes} [Fri Sep 23 15:03:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:03:21.955989 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6268988f-ef5d-4577-b5ff-37226ae2b9d3 None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:03:21.956383 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6268988f-ef5d-4577-b5ff-37226ae2b9d3 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:03:21.956550 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6268988f-ef5d-4577-b5ff-37226ae2b9d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:03:21.975670 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-6268988f-ef5d-4577-b5ff-37226ae2b9d3 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:03:21.980591 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-6268988f-ef5d-4577-b5ff-37226ae2b9d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:03:22.047324 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 633/1297] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Sep 23 15:03:21 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 23 15:03:22.314587 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efd38ce-8be5-4082-bc0f-03ba4607aa97 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:03:22.419971 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efd38ce-8be5-4082-bc0f-03ba4607aa97 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:03:22.455029 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096f8692-3b22-4b53-8f71-befdf5a4e9bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:03:22.455673 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096f8692-3b22-4b53-8f71-befdf5a4e9bd None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:03:22.455949 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096f8692-3b22-4b53-8f71-befdf5a4e9bd None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:03:22.456257 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096f8692-3b22-4b53-8f71-befdf5a4e9bd None admin] PATH_INFO: `/v3/services/placement` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:03:22.456864 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096f8692-3b22-4b53-8f71-befdf5a4e9bd None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:03:22.458229 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096f8692-3b22-4b53-8f71-befdf5a4e9bd None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:03:22.467286 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-096f8692-3b22-4b53-8f71-befdf5a4e9bd None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 23 15:03:22.467286 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:03:22.467286 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 23 15:03:22.467286 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 23 15:03:22.467286 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 23 15:03:22.467286 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 23 15:03:22.467286 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 23 15:03:22.467286 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 23 15:03:22.467286 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 23 15:03:22.467286 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Sep 23 15:03:22.467286 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 23 15:03:22.467286 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Sep 23 15:03:22.467286 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:03:22.467286 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:03:22.467286 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:03:22.467286 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:03:22.467286 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:03:22.475351 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:03:22.475351 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:03:22.475351 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:03:22.475351 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:03:22.475351 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:03:22.475351 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:03:22.475351 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 23 15:03:22.475351 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 23 15:03:22.475351 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 23 15:03:22.475351 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 23 15:03:22.475351 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:03:22.475351 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:03:22.475351 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:03:22.475351 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:03:22.475351 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:03:22.475351 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:03:22.476556 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:03:22.476556 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Sep 23 15:03:22.476556 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:03:22.476556 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:03:22.476556 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:03:22.476556 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:03:22.476556 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:03:22.476556 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:03:22.476556 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:03:22.476556 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:03:22.476556 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 23 15:03:22.476556 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 23 15:03:22.476556 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 23 15:03:22.476556 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Sep 23 15:03:22.476556 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 665/1298] 10.222.0.55 () {56 vars in 1117 bytes} [Fri Sep 23 15:03:22 2022] GET /identity/v3/services/placement => generated 90 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:03:22.477646 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6268988f-ef5d-4577-b5ff-37226ae2b9d3 None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:03:22.494035 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:03:22.577249 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6268988f-ef5d-4577-b5ff-37226ae2b9d3 None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:03:22.610292 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bf29db-b15f-40ac-83dd-24825ffde3a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:03:22.611084 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bf29db-b15f-40ac-83dd-24825ffde3a5 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:03:22.611575 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bf29db-b15f-40ac-83dd-24825ffde3a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:03:22.612512 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bf29db-b15f-40ac-83dd-24825ffde3a5 None admin] PATH_INFO: `/v3/services` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:03:22.613141 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bf29db-b15f-40ac-83dd-24825ffde3a5 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:03:22.613483 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bf29db-b15f-40ac-83dd-24825ffde3a5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:03:22.614938 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bf29db-b15f-40ac-83dd-24825ffde3a5 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:03:22.620453 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 634/1299] 10.222.0.55 () {56 vars in 1116 bytes} [Fri Sep 23 15:03:22 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:03:22.626474 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096f8692-3b22-4b53-8f71-befdf5a4e9bd None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:03:22.659017 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:03:22.726404 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096f8692-3b22-4b53-8f71-befdf5a4e9bd None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:03:22.758893 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e193b2-1be3-47e9-8d48-0308795575de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:03:22.759488 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e193b2-1be3-47e9-8d48-0308795575de None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:03:22.759770 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e193b2-1be3-47e9-8d48-0308795575de None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:03:22.760065 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e193b2-1be3-47e9-8d48-0308795575de None admin] PATH_INFO: `/v3/endpoints` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:03:22.760746 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e193b2-1be3-47e9-8d48-0308795575de None admin] RBAC: Adding query filter params (interface=public, service_id=1834fa1a0db046e08a4a02a6bf59a830, region_id=RegionOne) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:03:22.761048 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e193b2-1be3-47e9-8d48-0308795575de None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:03:22.762381 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e193b2-1be3-47e9-8d48-0308795575de None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:03:22.767193 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 666/1300] 10.222.0.55 () {56 vars in 1251 bytes} [Fri Sep 23 15:03:22 2022] GET /identity/v3/endpoints?service_id=1834fa1a0db046e08a4a02a6bf59a830&interface=public®ion_id=RegionOne => generated 194 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:03:22.773372 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bf29db-b15f-40ac-83dd-24825ffde3a5 None admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:03:22.857598 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:03:22.875824 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bf29db-b15f-40ac-83dd-24825ffde3a5 None admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:03:22.909605 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9bda7b-5ac8-4972-8aa4-fecd49234a20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:03:22.910197 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9bda7b-5ac8-4972-8aa4-fecd49234a20 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:03:22.911245 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9bda7b-5ac8-4972-8aa4-fecd49234a20 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:03:22.911529 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9bda7b-5ac8-4972-8aa4-fecd49234a20 None admin] PATH_INFO: `/v3/services` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:03:22.912145 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9bda7b-5ac8-4972-8aa4-fecd49234a20 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:03:22.913671 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9bda7b-5ac8-4972-8aa4-fecd49234a20 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:03:22.920679 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 635/1301] 10.222.0.55 () {56 vars in 1087 bytes} [Fri Sep 23 15:03:22 2022] GET /identity/v3/services => generated 2191 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:03:25.107169 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c93a57-ee5a-4f7e-88ab-e89882325b90 None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:03:25.117972 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c93a57-ee5a-4f7e-88ab-e89882325b90 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:03:25.117972 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c93a57-ee5a-4f7e-88ab-e89882325b90 None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:03:25.117972 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 667/1302] 10.222.0.55 () {54 vars in 927 bytes} [Fri Sep 23 15:03:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:03:25.118272 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2358e9f-9a01-422c-865f-6ffe56cd75e7 None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:03:25.118272 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2358e9f-9a01-422c-865f-6ffe56cd75e7 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:03:25.118272 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2358e9f-9a01-422c-865f-6ffe56cd75e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:03:25.135464 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-e2358e9f-9a01-422c-865f-6ffe56cd75e7 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:03:25.136795 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-e2358e9f-9a01-422c-865f-6ffe56cd75e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:03:25.202078 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 636/1303] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Sep 23 15:03:25 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 23 15:03:25.473127 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c93a57-ee5a-4f7e-88ab-e89882325b90 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:03:25.569918 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:03:25.576697 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c93a57-ee5a-4f7e-88ab-e89882325b90 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:03:25.611000 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b7cbd5-6e71-4eda-9d84-d4f3d6ec581e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:03:25.638921 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b7cbd5-6e71-4eda-9d84-d4f3d6ec581e None admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:03:25.638921 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b7cbd5-6e71-4eda-9d84-d4f3d6ec581e None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:03:25.638921 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b7cbd5-6e71-4eda-9d84-d4f3d6ec581e None admin] PATH_INFO: `/v3/services/placement` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:03:25.638921 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b7cbd5-6e71-4eda-9d84-d4f3d6ec581e None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:03:25.638921 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b7cbd5-6e71-4eda-9d84-d4f3d6ec581e None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:03:25.638921 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application [None req-37b7cbd5-6e71-4eda-9d84-d4f3d6ec581e None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 23 15:03:25.638921 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:03:25.639549 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 23 15:03:25.639549 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 23 15:03:25.639549 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 23 15:03:25.639549 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 23 15:03:25.639549 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 23 15:03:25.639549 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 23 15:03:25.639549 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 23 15:03:25.639549 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Sep 23 15:03:25.639549 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 23 15:03:25.639549 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Sep 23 15:03:25.639549 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:03:25.639549 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:03:25.639549 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:03:25.639549 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:03:25.639549 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:03:25.639549 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:03:25.639549 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:03:25.640737 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:03:25.640737 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:03:25.640737 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:03:25.640737 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:03:25.640737 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 23 15:03:25.640737 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 23 15:03:25.640737 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 23 15:03:25.640737 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 23 15:03:25.640737 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:03:25.640737 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:03:25.640737 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:03:25.640737 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:03:25.640737 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:03:25.640737 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:03:25.640737 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:03:25.640737 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application with Lock( Sep 23 15:03:25.640737 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:03:25.641711 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:03:25.641711 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:03:25.641711 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:03:25.641711 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:03:25.641711 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:03:25.641711 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:03:25.641711 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:03:25.641711 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 23 15:03:25.641711 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 23 15:03:25.641711 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 23 15:03:25.641711 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: ERROR keystone.server.flask.application Sep 23 15:03:25.641711 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 668/1304] 10.222.0.55 () {56 vars in 1117 bytes} [Fri Sep 23 15:03:25 2022] GET /identity/v3/services/placement => generated 90 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:03:25.642442 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2358e9f-9a01-422c-865f-6ffe56cd75e7 None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:03:25.710403 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:03:25.738118 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2358e9f-9a01-422c-865f-6ffe56cd75e7 None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:03:25.767752 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c7e95e-3d5c-43f4-ace1-0c581e9f7df2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:03:25.772578 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c7e95e-3d5c-43f4-ace1-0c581e9f7df2 None admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:03:25.772578 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c7e95e-3d5c-43f4-ace1-0c581e9f7df2 None admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:03:25.772578 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c7e95e-3d5c-43f4-ace1-0c581e9f7df2 None admin] PATH_INFO: `/v3/services` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:03:25.772578 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c7e95e-3d5c-43f4-ace1-0c581e9f7df2 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:03:25.772578 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c7e95e-3d5c-43f4-ace1-0c581e9f7df2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:03:25.772578 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c7e95e-3d5c-43f4-ace1-0c581e9f7df2 None admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:03:25.777644 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 637/1305] 10.222.0.55 () {56 vars in 1116 bytes} [Fri Sep 23 15:03:25 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:03:25.784843 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b7cbd5-6e71-4eda-9d84-d4f3d6ec581e None admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:03:25.890657 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b7cbd5-6e71-4eda-9d84-d4f3d6ec581e None admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:03:25.928758 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af0f24c-d213-41ed-ad22-0caa8a1ee07b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:03:25.929384 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af0f24c-d213-41ed-ad22-0caa8a1ee07b None admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:03:25.929665 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af0f24c-d213-41ed-ad22-0caa8a1ee07b None admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:03:25.929938 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af0f24c-d213-41ed-ad22-0caa8a1ee07b None admin] PATH_INFO: `/v3/endpoints` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:03:25.930665 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af0f24c-d213-41ed-ad22-0caa8a1ee07b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:03:25.932057 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af0f24c-d213-41ed-ad22-0caa8a1ee07b None admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:03:26.059234 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 669/1306] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Sep 23 15:03:25 2022] POST /identity/v3/endpoints => generated 331 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 23 15:03:35.829134 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=30 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:03:36.054141 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:11:19.916570 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cfdb4a-dd0b-418d-bd69-4bdf7f25a1f9 None None] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:11:19.917043 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cfdb4a-dd0b-418d-bd69-4bdf7f25a1f9 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:11:19.917043 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cfdb4a-dd0b-418d-bd69-4bdf7f25a1f9 None None] PATH_INFO: `/` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:11:19.924773 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5e1edc-5778-4e67-b564-62f44e34e137 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:11:19.925211 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5e1edc-5778-4e67-b564-62f44e34e137 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:11:19.926036 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5e1edc-5778-4e67-b564-62f44e34e137 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:11:19.930208 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 638/1307] 10.222.0.55 () {54 vars in 927 bytes} [Fri Sep 23 15:11:19 2022] GET /identity => generated 270 bytes in 15 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:11:20.068845 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-ec5e1edc-5778-4e67-b564-62f44e34e137 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:11:20.070315 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-ec5e1edc-5778-4e67-b564-62f44e34e137 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:11:20.228012 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 670/1308] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Sep 23 15:11:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 291 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 15:11:20.537942 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e1df49-8cd9-4b4b-a69f-246af898d510 None None] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:11:20.538573 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e1df49-8cd9-4b4b-a69f-246af898d510 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:11:20.539059 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e1df49-8cd9-4b4b-a69f-246af898d510 None None] PATH_INFO: `/` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:11:20.540916 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 639/1309] 10.222.0.55 () {54 vars in 964 bytes} [Fri Sep 23 15:11:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:11:20.546744 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150aa202-7ddf-4a55-90af-adf288e80b76 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:11:20.547031 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150aa202-7ddf-4a55-90af-adf288e80b76 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:11:20.547312 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150aa202-7ddf-4a55-90af-adf288e80b76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:11:20.644995 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-150aa202-7ddf-4a55-90af-adf288e80b76 None None] MFA Rules not processed for user `d259178570864ffeb936ee10dfec6b28`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:11:20.646305 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-150aa202-7ddf-4a55-90af-adf288e80b76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:11:20.675151 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:11:20.754748 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 671/1310] 10.222.0.55 () {58 vars in 1061 bytes} [Fri Sep 23 15:11:20 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:11:20.767942 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e1df49-8cd9-4b4b-a69f-246af898d510 None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:11:20.770308 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-13e1df49-8cd9-4b4b-a69f-246af898d510 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:11:20.902165 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e1df49-8cd9-4b4b-a69f-246af898d510 None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:11:20.989815 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aed204e-7cfd-4134-80d0-4c8b0b014b0d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd259178570864ffeb936ee10dfec6b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:11:20.990694 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aed204e-7cfd-4134-80d0-4c8b0b014b0d service neutron] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:11:20.990946 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aed204e-7cfd-4134-80d0-4c8b0b014b0d service neutron] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:11:20.991296 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aed204e-7cfd-4134-80d0-4c8b0b014b0d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:11:20.993399 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-3aed204e-7cfd-4134-80d0-4c8b0b014b0d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:11:21.060543 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:11:21.138247 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aed204e-7cfd-4134-80d0-4c8b0b014b0d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:11:21.140073 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aed204e-7cfd-4134-80d0-4c8b0b014b0d service neutron] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:11:21.177510 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 640/1311] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:11:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:11:23.685055 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d7195f-d529-436d-82c7-f5eb50be7483 None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:11:23.685585 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d7195f-d529-436d-82c7-f5eb50be7483 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:11:23.685676 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d7195f-d529-436d-82c7-f5eb50be7483 None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:11:23.687014 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 672/1312] 10.222.0.55 () {54 vars in 927 bytes} [Fri Sep 23 15:11:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:11:23.692830 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f72d7b9-f4fa-4218-b3ad-90dfe2ca076d None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:11:23.693280 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f72d7b9-f4fa-4218-b3ad-90dfe2ca076d None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:11:23.693379 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f72d7b9-f4fa-4218-b3ad-90dfe2ca076d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:11:23.764359 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-8f72d7b9-f4fa-4218-b3ad-90dfe2ca076d None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:11:23.765496 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-8f72d7b9-f4fa-4218-b3ad-90dfe2ca076d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:11:23.863489 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 641/1313] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Sep 23 15:11:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:11:24.152931 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d7195f-d529-436d-82c7-f5eb50be7483 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:11:24.154484 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-61d7195f-d529-436d-82c7-f5eb50be7483 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:11:24.268829 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d7195f-d529-436d-82c7-f5eb50be7483 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:11:24.342953 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:11:24.345712 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad61785f-5c6e-471c-9ebb-ceaaa9c95a64 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd259178570864ffeb936ee10dfec6b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:11:24.346334 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad61785f-5c6e-471c-9ebb-ceaaa9c95a64 service neutron] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:11:24.346613 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad61785f-5c6e-471c-9ebb-ceaaa9c95a64 service neutron] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:11:24.346912 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad61785f-5c6e-471c-9ebb-ceaaa9c95a64 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:11:24.348614 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-ad61785f-5c6e-471c-9ebb-ceaaa9c95a64 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:11:24.467730 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad61785f-5c6e-471c-9ebb-ceaaa9c95a64 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:11:24.469837 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad61785f-5c6e-471c-9ebb-ceaaa9c95a64 service neutron] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:11:24.489919 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 673/1314] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:11:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:11:27.304426 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1715dc-a155-4c06-9c68-9d3c46aa7f4a None None] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:11:27.305033 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1715dc-a155-4c06-9c68-9d3c46aa7f4a None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:11:27.305033 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1715dc-a155-4c06-9c68-9d3c46aa7f4a None None] PATH_INFO: `/` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:11:27.306299 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 642/1315] 10.222.0.55 () {54 vars in 927 bytes} [Fri Sep 23 15:11:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:11:27.311554 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf25904c-40b0-4177-b5ab-ee2040715823 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:11:27.311964 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf25904c-40b0-4177-b5ab-ee2040715823 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:11:27.316647 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf25904c-40b0-4177-b5ab-ee2040715823 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:11:27.383818 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-bf25904c-40b0-4177-b5ab-ee2040715823 None None] MFA Rules not processed for user `ca9091e894bc4e01ab24bbbb474a09cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:11:27.384960 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-bf25904c-40b0-4177-b5ab-ee2040715823 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:11:27.440323 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:11:27.484492 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 674/1316] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Sep 23 15:11:27 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:11:27.769914 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1715dc-a155-4c06-9c68-9d3c46aa7f4a None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:11:27.772470 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-cd1715dc-a155-4c06-9c68-9d3c46aa7f4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:11:27.784592 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:11:27.882389 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1715dc-a155-4c06-9c68-9d3c46aa7f4a None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:11:27.957612 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83797b3-a142-4e29-abff-f975714b00b7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd259178570864ffeb936ee10dfec6b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:11:27.958221 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83797b3-a142-4e29-abff-f975714b00b7 service neutron] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:11:27.958500 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83797b3-a142-4e29-abff-f975714b00b7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:11:27.958766 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83797b3-a142-4e29-abff-f975714b00b7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:11:27.960391 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-b83797b3-a142-4e29-abff-f975714b00b7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:11:28.022168 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:11:28.069044 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83797b3-a142-4e29-abff-f975714b00b7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:11:28.070480 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83797b3-a142-4e29-abff-f975714b00b7 service neutron] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:11:28.089956 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 643/1317] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:11:27 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:11:31.090409 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e4e55d-af6d-48fa-9dd7-a225f25466ee None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:11:31.090900 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e4e55d-af6d-48fa-9dd7-a225f25466ee None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:11:31.091016 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e4e55d-af6d-48fa-9dd7-a225f25466ee None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:11:31.092337 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 675/1318] 10.222.0.55 () {54 vars in 927 bytes} [Fri Sep 23 15:11:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:11:31.098575 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bba2ff-affc-414a-883c-6045724339da None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:11:31.099003 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bba2ff-affc-414a-883c-6045724339da None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:11:31.101851 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bba2ff-affc-414a-883c-6045724339da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:11:31.171864 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-a7bba2ff-affc-414a-883c-6045724339da None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:11:31.173716 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-a7bba2ff-affc-414a-883c-6045724339da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:11:31.277975 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 644/1319] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Sep 23 15:11:31 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:11:31.585291 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcb2f0f-fbea-43df-bdb7-38a14601490a None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:11:31.585565 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcb2f0f-fbea-43df-bdb7-38a14601490a None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:11:31.585834 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcb2f0f-fbea-43df-bdb7-38a14601490a None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:11:31.587167 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 676/1320] 10.222.0.55 () {54 vars in 964 bytes} [Fri Sep 23 15:11:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 23 15:11:31.593175 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b066750-f790-4534-a571-6412c328c692 None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:11:31.593461 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b066750-f790-4534-a571-6412c328c692 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:11:31.593737 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b066750-f790-4534-a571-6412c328c692 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:11:31.619781 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:11:31.682158 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-6b066750-f790-4534-a571-6412c328c692 None None] MFA Rules not processed for user `d259178570864ffeb936ee10dfec6b28`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:11:31.683369 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-6b066750-f790-4534-a571-6412c328c692 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:11:31.760243 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 645/1321] 10.222.0.55 () {58 vars in 1061 bytes} [Fri Sep 23 15:11:31 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:11:31.771189 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcb2f0f-fbea-43df-bdb7-38a14601490a None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:11:31.772706 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-4dcb2f0f-fbea-43df-bdb7-38a14601490a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:11:31.884565 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcb2f0f-fbea-43df-bdb7-38a14601490a None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:11:31.964529 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942b3e38-c647-4b6f-88c7-7f50aceeaa29 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd259178570864ffeb936ee10dfec6b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:11:31.965179 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942b3e38-c647-4b6f-88c7-7f50aceeaa29 service neutron] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:11:31.965464 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942b3e38-c647-4b6f-88c7-7f50aceeaa29 service neutron] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:11:31.965739 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942b3e38-c647-4b6f-88c7-7f50aceeaa29 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:11:31.967364 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-942b3e38-c647-4b6f-88c7-7f50aceeaa29 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:11:31.975558 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:11:32.089671 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942b3e38-c647-4b6f-88c7-7f50aceeaa29 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:11:32.091138 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942b3e38-c647-4b6f-88c7-7f50aceeaa29 service neutron] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:11:32.112360 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 677/1322] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:11:31 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:11:35.845402 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82cd675-595c-4113-8c40-0626df3b0488 None None] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:11:35.845994 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82cd675-595c-4113-8c40-0626df3b0488 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:11:35.845994 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82cd675-595c-4113-8c40-0626df3b0488 None None] PATH_INFO: `/` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:11:35.847221 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 646/1323] 10.222.0.55 () {54 vars in 927 bytes} [Fri Sep 23 15:11:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:11:35.853260 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27db700a-c917-4b7f-a988-45dfa7eb1b8b None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:11:35.853773 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27db700a-c917-4b7f-a988-45dfa7eb1b8b None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:11:35.853944 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27db700a-c917-4b7f-a988-45dfa7eb1b8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:11:35.924214 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-27db700a-c917-4b7f-a988-45dfa7eb1b8b None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:11:35.925436 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-27db700a-c917-4b7f-a988-45dfa7eb1b8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:11:35.987756 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:11:36.030675 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 678/1324] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Sep 23 15:11:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:11:36.319151 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82cd675-595c-4113-8c40-0626df3b0488 None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:11:36.320732 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-a82cd675-595c-4113-8c40-0626df3b0488 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:11:36.449000 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82cd675-595c-4113-8c40-0626df3b0488 None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:11:36.464266 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:11:36.536531 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f733f38b-980f-4518-a3db-d0d047e73dce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd259178570864ffeb936ee10dfec6b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:11:36.537146 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f733f38b-980f-4518-a3db-d0d047e73dce service neutron] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:11:36.537418 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f733f38b-980f-4518-a3db-d0d047e73dce service neutron] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:11:36.537684 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f733f38b-980f-4518-a3db-d0d047e73dce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:11:36.539547 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-f733f38b-980f-4518-a3db-d0d047e73dce service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:11:36.660011 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f733f38b-980f-4518-a3db-d0d047e73dce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:11:36.661380 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f733f38b-980f-4518-a3db-d0d047e73dce service neutron] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:11:36.680076 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 647/1325] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:11:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 23 15:11:39.026704 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba2fc5b-db81-4039-bdc8-781043a0ea75 None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:11:39.027313 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba2fc5b-db81-4039-bdc8-781043a0ea75 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:11:39.027313 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba2fc5b-db81-4039-bdc8-781043a0ea75 None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:11:39.028660 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 679/1326] 10.222.0.55 () {54 vars in 927 bytes} [Fri Sep 23 15:11:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:11:39.035558 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a9970a-bf67-4b50-9a73-f0340f2a4dc9 None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:11:39.036103 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a9970a-bf67-4b50-9a73-f0340f2a4dc9 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:11:39.036196 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a9970a-bf67-4b50-9a73-f0340f2a4dc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:11:39.091218 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:11:39.111851 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-12a9970a-bf67-4b50-9a73-f0340f2a4dc9 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:11:39.114552 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-12a9970a-bf67-4b50-9a73-f0340f2a4dc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:11:39.225952 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 648/1327] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Sep 23 15:11:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:11:39.524065 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba2fc5b-db81-4039-bdc8-781043a0ea75 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:11:39.525681 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-5ba2fc5b-db81-4039-bdc8-781043a0ea75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:11:39.648688 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba2fc5b-db81-4039-bdc8-781043a0ea75 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:11:39.726575 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efe558b-1baa-4346-8bb3-c1668d1cac04 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd259178570864ffeb936ee10dfec6b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:11:39.727186 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efe558b-1baa-4346-8bb3-c1668d1cac04 service neutron] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:11:39.727464 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efe558b-1baa-4346-8bb3-c1668d1cac04 service neutron] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:11:39.727734 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efe558b-1baa-4346-8bb3-c1668d1cac04 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:11:39.729388 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-2efe558b-1baa-4346-8bb3-c1668d1cac04 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:11:39.747178 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:11:39.842167 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efe558b-1baa-4346-8bb3-c1668d1cac04 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:11:39.843463 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efe558b-1baa-4346-8bb3-c1668d1cac04 service neutron] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:11:39.864008 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 680/1328] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:11:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:11:42.122425 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb14039c-2a7d-49d8-8165-aaadddf4b5ca None None] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:11:42.122931 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb14039c-2a7d-49d8-8165-aaadddf4b5ca None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:11:42.123056 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb14039c-2a7d-49d8-8165-aaadddf4b5ca None None] PATH_INFO: `/` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:11:42.124329 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 649/1329] 10.222.0.55 () {54 vars in 927 bytes} [Fri Sep 23 15:11:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:11:42.129521 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad4a175-a0dc-4a8f-9dc8-2f6de2a7ed46 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:11:42.129988 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad4a175-a0dc-4a8f-9dc8-2f6de2a7ed46 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:11:42.130310 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad4a175-a0dc-4a8f-9dc8-2f6de2a7ed46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:11:42.200870 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-1ad4a175-a0dc-4a8f-9dc8-2f6de2a7ed46 None None] MFA Rules not processed for user `ca9091e894bc4e01ab24bbbb474a09cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:11:42.202098 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-1ad4a175-a0dc-4a8f-9dc8-2f6de2a7ed46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:11:42.294996 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:11:42.338750 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 681/1330] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Sep 23 15:11:42 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:11:42.645656 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb14039c-2a7d-49d8-8165-aaadddf4b5ca None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:11:42.647191 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-eb14039c-2a7d-49d8-8165-aaadddf4b5ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:11:42.753994 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb14039c-2a7d-49d8-8165-aaadddf4b5ca None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:11:42.776240 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:11:42.840479 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add3bbd1-1627-432a-af60-8c093056470b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd259178570864ffeb936ee10dfec6b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:11:42.841103 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add3bbd1-1627-432a-af60-8c093056470b service neutron] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:11:42.841389 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add3bbd1-1627-432a-af60-8c093056470b service neutron] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:11:42.841663 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add3bbd1-1627-432a-af60-8c093056470b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:11:42.843322 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-add3bbd1-1627-432a-af60-8c093056470b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:11:42.976410 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add3bbd1-1627-432a-af60-8c093056470b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:11:42.977875 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add3bbd1-1627-432a-af60-8c093056470b service neutron] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:11:42.998339 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 650/1331] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:11:42 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:11:45.741944 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcb1bc0-873f-4bc3-91b2-a19316365314 None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:11:45.742461 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcb1bc0-873f-4bc3-91b2-a19316365314 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:11:45.742461 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcb1bc0-873f-4bc3-91b2-a19316365314 None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:11:45.743861 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 682/1332] 10.222.0.55 () {54 vars in 927 bytes} [Fri Sep 23 15:11:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:11:45.749362 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55a76ae-bad2-4261-a508-1de1c602fd75 None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:11:45.749771 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55a76ae-bad2-4261-a508-1de1c602fd75 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:11:45.749916 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55a76ae-bad2-4261-a508-1de1c602fd75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:11:45.818333 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:11:45.821004 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-a55a76ae-bad2-4261-a508-1de1c602fd75 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:11:45.822222 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-a55a76ae-bad2-4261-a508-1de1c602fd75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:11:45.922887 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 651/1333] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Sep 23 15:11:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:11:46.208261 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcb1bc0-873f-4bc3-91b2-a19316365314 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:11:46.209847 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-8dcb1bc0-873f-4bc3-91b2-a19316365314 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:11:46.316770 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcb1bc0-873f-4bc3-91b2-a19316365314 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:11:46.397248 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130c639f-b2db-4628-aee5-2acb68f62145 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd259178570864ffeb936ee10dfec6b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:11:46.397703 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130c639f-b2db-4628-aee5-2acb68f62145 service neutron] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:11:46.397983 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130c639f-b2db-4628-aee5-2acb68f62145 service neutron] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:11:46.398255 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130c639f-b2db-4628-aee5-2acb68f62145 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:11:46.399876 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-130c639f-b2db-4628-aee5-2acb68f62145 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:11:46.428884 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:11:46.514166 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130c639f-b2db-4628-aee5-2acb68f62145 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:11:46.515587 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130c639f-b2db-4628-aee5-2acb68f62145 service neutron] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:11:46.536163 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 683/1334] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:11:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:11:48.777979 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842fd67e-7478-4516-8dc7-81e369c62d38 None None] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:11:48.778546 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842fd67e-7478-4516-8dc7-81e369c62d38 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:11:48.778546 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842fd67e-7478-4516-8dc7-81e369c62d38 None None] PATH_INFO: `/` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:11:48.779909 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 652/1335] 10.222.0.55 () {54 vars in 927 bytes} [Fri Sep 23 15:11:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:11:48.786655 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff365456-b51a-4779-a2a3-d8466866ce9f None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:11:48.787194 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff365456-b51a-4779-a2a3-d8466866ce9f None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:11:48.787194 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff365456-b51a-4779-a2a3-d8466866ce9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:11:48.855901 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-ff365456-b51a-4779-a2a3-d8466866ce9f None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:11:48.857220 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-ff365456-b51a-4779-a2a3-d8466866ce9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:11:48.933526 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:11:48.956287 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 684/1336] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Sep 23 15:11:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:11:49.240110 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842fd67e-7478-4516-8dc7-81e369c62d38 None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:11:49.241673 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-842fd67e-7478-4516-8dc7-81e369c62d38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:11:49.349073 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842fd67e-7478-4516-8dc7-81e369c62d38 None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:11:49.380400 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:11:49.427455 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7abaa5-3f63-4e98-9461-df88850ea715 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd259178570864ffeb936ee10dfec6b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:11:49.428100 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7abaa5-3f63-4e98-9461-df88850ea715 service neutron] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:11:49.428387 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7abaa5-3f63-4e98-9461-df88850ea715 service neutron] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:11:49.428658 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7abaa5-3f63-4e98-9461-df88850ea715 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:11:49.430258 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-ea7abaa5-3f63-4e98-9461-df88850ea715 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:11:49.542194 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7abaa5-3f63-4e98-9461-df88850ea715 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:11:49.543639 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7abaa5-3f63-4e98-9461-df88850ea715 service neutron] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:11:49.562376 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 653/1337] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:11:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:11:52.440568 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55219646-6015-4367-88c6-27b200c4996b None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:11:52.441143 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55219646-6015-4367-88c6-27b200c4996b None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:11:52.441143 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55219646-6015-4367-88c6-27b200c4996b None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:11:52.442461 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 685/1338] 10.222.0.55 () {54 vars in 927 bytes} [Fri Sep 23 15:11:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:11:52.447671 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db66bdea-bdb0-485c-960b-926c5a8c63d5 None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:11:52.448152 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db66bdea-bdb0-485c-960b-926c5a8c63d5 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:11:52.448295 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db66bdea-bdb0-485c-960b-926c5a8c63d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:11:52.516498 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-db66bdea-bdb0-485c-960b-926c5a8c63d5 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:11:52.518152 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-db66bdea-bdb0-485c-960b-926c5a8c63d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:11:52.526908 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:11:52.617732 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 654/1339] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Sep 23 15:11:52 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:11:52.900619 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55219646-6015-4367-88c6-27b200c4996b None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:11:52.902186 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-55219646-6015-4367-88c6-27b200c4996b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:11:53.007144 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55219646-6015-4367-88c6-27b200c4996b None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:11:53.088772 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf34e533-24ca-4570-adcf-d828e759266b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd259178570864ffeb936ee10dfec6b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:11:53.089414 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf34e533-24ca-4570-adcf-d828e759266b service neutron] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:11:53.089693 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf34e533-24ca-4570-adcf-d828e759266b service neutron] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:11:53.089960 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf34e533-24ca-4570-adcf-d828e759266b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:11:53.091537 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-cf34e533-24ca-4570-adcf-d828e759266b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:11:53.127419 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:11:53.202207 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf34e533-24ca-4570-adcf-d828e759266b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:11:53.204388 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf34e533-24ca-4570-adcf-d828e759266b service neutron] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:11:53.224410 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 686/1340] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:11:52 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:11:57.740666 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa9a229-eda0-4ce1-9b0d-e7ee8fec807d None None] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:11:57.741154 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa9a229-eda0-4ce1-9b0d-e7ee8fec807d None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:11:57.741310 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa9a229-eda0-4ce1-9b0d-e7ee8fec807d None None] PATH_INFO: `/` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:11:57.742756 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 655/1341] 10.222.0.55 () {54 vars in 927 bytes} [Fri Sep 23 15:11:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:11:57.749738 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243505b4-00f0-4037-8f66-77f69b2ea167 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:11:57.750704 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243505b4-00f0-4037-8f66-77f69b2ea167 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:11:57.751009 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243505b4-00f0-4037-8f66-77f69b2ea167 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:11:57.829635 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-243505b4-00f0-4037-8f66-77f69b2ea167 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:11:57.830793 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-243505b4-00f0-4037-8f66-77f69b2ea167 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:11:57.924976 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:11:57.935909 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 687/1342] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Sep 23 15:11:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:11:58.233659 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa9a229-eda0-4ce1-9b0d-e7ee8fec807d None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:11:58.235957 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-0aa9a229-eda0-4ce1-9b0d-e7ee8fec807d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:11:58.347959 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa9a229-eda0-4ce1-9b0d-e7ee8fec807d None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:11:58.395052 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:11:58.432714 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2243845f-9b26-4d10-9d00-1ef0e0f5b4c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd259178570864ffeb936ee10dfec6b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:11:58.433330 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2243845f-9b26-4d10-9d00-1ef0e0f5b4c9 service neutron] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:11:58.433611 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2243845f-9b26-4d10-9d00-1ef0e0f5b4c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:11:58.433886 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2243845f-9b26-4d10-9d00-1ef0e0f5b4c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:11:58.435615 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-2243845f-9b26-4d10-9d00-1ef0e0f5b4c9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:11:58.548657 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2243845f-9b26-4d10-9d00-1ef0e0f5b4c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:11:58.550113 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2243845f-9b26-4d10-9d00-1ef0e0f5b4c9 service neutron] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:11:58.569603 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 656/1343] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:11:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:12:01.790988 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4542aa-5973-48ce-9e19-1d5034f2fb24 None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:12:01.791516 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4542aa-5973-48ce-9e19-1d5034f2fb24 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:12:01.791516 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4542aa-5973-48ce-9e19-1d5034f2fb24 None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:12:01.792862 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 688/1344] 10.222.0.55 () {54 vars in 927 bytes} [Fri Sep 23 15:12:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:12:01.798848 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d499df6-56a3-452f-8a9a-bcdfb7bc5494 None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:12:01.799381 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d499df6-56a3-452f-8a9a-bcdfb7bc5494 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:12:01.799514 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d499df6-56a3-452f-8a9a-bcdfb7bc5494 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:12:01.868044 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-8d499df6-56a3-452f-8a9a-bcdfb7bc5494 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:12:01.869280 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-8d499df6-56a3-452f-8a9a-bcdfb7bc5494 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:12:01.890274 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:12:01.973260 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 657/1345] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Sep 23 15:12:01 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 15:12:02.274481 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4542aa-5973-48ce-9e19-1d5034f2fb24 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:12:02.276478 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-4e4542aa-5973-48ce-9e19-1d5034f2fb24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:12:02.389306 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4542aa-5973-48ce-9e19-1d5034f2fb24 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:12:02.468483 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea13127b-8bc0-4704-958d-168c4b510d96 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd259178570864ffeb936ee10dfec6b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:12:02.469782 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea13127b-8bc0-4704-958d-168c4b510d96 service neutron] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:12:02.470144 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea13127b-8bc0-4704-958d-168c4b510d96 service neutron] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:12:02.470428 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea13127b-8bc0-4704-958d-168c4b510d96 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:12:02.472222 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-ea13127b-8bc0-4704-958d-168c4b510d96 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:12:02.546125 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:12:02.621594 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea13127b-8bc0-4704-958d-168c4b510d96 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:12:02.622901 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea13127b-8bc0-4704-958d-168c4b510d96 service neutron] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:12:02.647095 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 689/1346] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:12:02 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:12:07.304248 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8899e9a-336c-4ea1-a044-c474faffb6a5 None None] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:12:07.304825 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8899e9a-336c-4ea1-a044-c474faffb6a5 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:12:07.304825 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8899e9a-336c-4ea1-a044-c474faffb6a5 None None] PATH_INFO: `/` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:12:07.306143 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 658/1347] 10.222.0.55 () {54 vars in 927 bytes} [Fri Sep 23 15:12:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:12:07.311839 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db6256d-513e-4d00-a7ae-adedecb62a0f None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:12:07.312325 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db6256d-513e-4d00-a7ae-adedecb62a0f None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:12:07.312583 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db6256d-513e-4d00-a7ae-adedecb62a0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:12:07.388060 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-2db6256d-513e-4d00-a7ae-adedecb62a0f None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:12:07.389293 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-2db6256d-513e-4d00-a7ae-adedecb62a0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:12:07.493873 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:12:07.496030 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 690/1348] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Sep 23 15:12:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:12:07.780647 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8899e9a-336c-4ea1-a044-c474faffb6a5 None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:12:07.786446 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-d8899e9a-336c-4ea1-a044-c474faffb6a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:12:07.908267 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8899e9a-336c-4ea1-a044-c474faffb6a5 None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:12:07.963572 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:12:07.992257 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f048a5-1bb1-4ed7-9407-c791a8ea9183 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd259178570864ffeb936ee10dfec6b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:12:07.992885 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f048a5-1bb1-4ed7-9407-c791a8ea9183 service neutron] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:12:07.993166 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f048a5-1bb1-4ed7-9407-c791a8ea9183 service neutron] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:12:07.993434 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f048a5-1bb1-4ed7-9407-c791a8ea9183 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:12:07.995082 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-58f048a5-1bb1-4ed7-9407-c791a8ea9183 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:12:08.111860 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f048a5-1bb1-4ed7-9407-c791a8ea9183 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:12:08.113361 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f048a5-1bb1-4ed7-9407-c791a8ea9183 service neutron] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:12:08.132806 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 659/1349] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:12:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:12:13.167710 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f44a562-ee26-4c0d-84c2-43246548b1fd None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:12:13.168242 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f44a562-ee26-4c0d-84c2-43246548b1fd None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:12:13.168505 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f44a562-ee26-4c0d-84c2-43246548b1fd None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:12:13.170159 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 691/1350] 10.222.0.55 () {54 vars in 921 bytes} [Fri Sep 23 15:12:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:12:13.176728 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a728af-a4e2-4782-a955-d36336acd1a7 None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:12:13.177224 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a728af-a4e2-4782-a955-d36336acd1a7 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:12:13.177554 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a728af-a4e2-4782-a955-d36336acd1a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:12:13.282260 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-22a728af-a4e2-4782-a955-d36336acd1a7 None None] MFA Rules not processed for user `09f1a55d180642c781a6e0b017d9f0aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:12:13.283591 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-22a728af-a4e2-4782-a955-d36336acd1a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:12:13.304659 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:12:13.478907 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 660/1351] 10.222.0.55 () {58 vars in 1018 bytes} [Fri Sep 23 15:12:13 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 303 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:12:17.771174 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32ed438-40df-48ea-a102-7ceac312e5b3 None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:12:17.771798 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32ed438-40df-48ea-a102-7ceac312e5b3 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:12:17.771798 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32ed438-40df-48ea-a102-7ceac312e5b3 None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:12:17.773358 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 692/1352] 10.222.0.55 () {54 vars in 921 bytes} [Fri Sep 23 15:12:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:12:17.778525 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6978e0d-6fbc-44b1-96dc-2d494c303142 None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:12:17.778966 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6978e0d-6fbc-44b1-96dc-2d494c303142 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:12:17.779079 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6978e0d-6fbc-44b1-96dc-2d494c303142 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:12:17.868903 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-b6978e0d-6fbc-44b1-96dc-2d494c303142 None None] MFA Rules not processed for user `09f1a55d180642c781a6e0b017d9f0aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:12:17.870043 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-b6978e0d-6fbc-44b1-96dc-2d494c303142 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:12:17.954445 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:12:18.005846 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 661/1353] 10.222.0.55 () {58 vars in 1018 bytes} [Fri Sep 23 15:12:17 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:12:19.557891 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336b280c-7d34-498f-8b1b-864c75a51c42 None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:12:19.558332 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336b280c-7d34-498f-8b1b-864c75a51c42 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:12:19.558440 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336b280c-7d34-498f-8b1b-864c75a51c42 None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:12:19.559818 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 693/1354] 10.222.0.55 () {54 vars in 921 bytes} [Fri Sep 23 15:12:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:12:19.565482 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31604fbb-0fed-42e1-81e9-c94ece908ce2 None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:12:19.565970 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31604fbb-0fed-42e1-81e9-c94ece908ce2 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:12:19.566091 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31604fbb-0fed-42e1-81e9-c94ece908ce2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:12:19.649785 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-31604fbb-0fed-42e1-81e9-c94ece908ce2 None None] MFA Rules not processed for user `09f1a55d180642c781a6e0b017d9f0aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:12:19.651009 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-31604fbb-0fed-42e1-81e9-c94ece908ce2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:12:19.771003 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 662/1355] 10.222.0.55 () {58 vars in 1018 bytes} [Fri Sep 23 15:12:19 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:12:23.661466 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c8b4b0-3545-4df6-bf10-3862bbe11ffa None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:12:23.662654 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c8b4b0-3545-4df6-bf10-3862bbe11ffa None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:12:23.663234 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c8b4b0-3545-4df6-bf10-3862bbe11ffa None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:12:23.665201 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 694/1356] 10.222.0.55 () {54 vars in 927 bytes} [Fri Sep 23 15:12:23 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:12:23.672490 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae02d08-d5c3-444e-a2d9-7bb326c050b8 None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:12:23.673629 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae02d08-d5c3-444e-a2d9-7bb326c050b8 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:12:23.674137 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae02d08-d5c3-444e-a2d9-7bb326c050b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:12:23.721641 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:12:23.760608 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-8ae02d08-d5c3-444e-a2d9-7bb326c050b8 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:12:23.762963 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-8ae02d08-d5c3-444e-a2d9-7bb326c050b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:12:23.904884 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 663/1357] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Sep 23 15:12:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:12:24.341345 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ebf94b-254b-4ade-8cc8-8f593d5fdccd None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:12:24.342182 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ebf94b-254b-4ade-8cc8-8f593d5fdccd None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:12:24.342939 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ebf94b-254b-4ade-8cc8-8f593d5fdccd None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:12:24.346747 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 695/1358] 10.222.0.55 () {54 vars in 961 bytes} [Fri Sep 23 15:12:24 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:12:24.355049 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d819129d-52c9-4629-8738-e33b969830f8 None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:12:24.355807 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d819129d-52c9-4629-8738-e33b969830f8 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:12:24.357786 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d819129d-52c9-4629-8738-e33b969830f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:12:24.487564 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-d819129d-52c9-4629-8738-e33b969830f8 None None] MFA Rules not processed for user `b3c6c7a1325a47be924268e1aa1a7a18`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:12:24.489291 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-d819129d-52c9-4629-8738-e33b969830f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:12:24.564766 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:12:24.657104 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 664/1359] 10.222.0.55 () {58 vars in 1058 bytes} [Fri Sep 23 15:12:24 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 303 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:12:24.666477 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ebf94b-254b-4ade-8cc8-8f593d5fdccd None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:12:24.668677 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-27ebf94b-254b-4ade-8cc8-8f593d5fdccd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:12:24.886413 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ebf94b-254b-4ade-8cc8-8f593d5fdccd None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:12:24.988396 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:12:25.047655 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c3ebeb-7cd2-479f-845e-622760361966 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3c6c7a1325a47be924268e1aa1a7a18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:12:25.048784 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c3ebeb-7cd2-479f-845e-622760361966 service nova] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:12:25.049219 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c3ebeb-7cd2-479f-845e-622760361966 service nova] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:12:25.049659 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c3ebeb-7cd2-479f-845e-622760361966 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:12:25.051795 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-99c3ebeb-7cd2-479f-845e-622760361966 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:12:25.176495 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c3ebeb-7cd2-479f-845e-622760361966 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:12:25.178826 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c3ebeb-7cd2-479f-845e-622760361966 service nova] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:12:25.205234 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 696/1360] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:12:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 539 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:12:28.207116 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95aadca6-e9f7-4409-bb8a-a28a16ea5043 None None] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:12:28.208014 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95aadca6-e9f7-4409-bb8a-a28a16ea5043 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:12:28.208450 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95aadca6-e9f7-4409-bb8a-a28a16ea5043 None None] PATH_INFO: `/` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:12:28.209969 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 665/1361] 10.222.0.55 () {54 vars in 927 bytes} [Fri Sep 23 15:12:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:12:28.216023 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342111c3-99f3-48e4-87a5-607ef46bcf04 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:12:28.216857 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342111c3-99f3-48e4-87a5-607ef46bcf04 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:12:28.217141 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342111c3-99f3-48e4-87a5-607ef46bcf04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:12:28.293507 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-342111c3-99f3-48e4-87a5-607ef46bcf04 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:12:28.294676 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-342111c3-99f3-48e4-87a5-607ef46bcf04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:12:28.335544 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:12:28.411528 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 697/1362] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Sep 23 15:12:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:12:28.711061 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95aadca6-e9f7-4409-bb8a-a28a16ea5043 None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:12:28.712853 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-95aadca6-e9f7-4409-bb8a-a28a16ea5043 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:12:28.878902 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95aadca6-e9f7-4409-bb8a-a28a16ea5043 None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:12:28.889055 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:12:29.047345 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cde8ca-96c0-4a60-887a-4abfe0f4adb0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3c6c7a1325a47be924268e1aa1a7a18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:12:29.048416 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cde8ca-96c0-4a60-887a-4abfe0f4adb0 service nova] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:12:29.048690 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cde8ca-96c0-4a60-887a-4abfe0f4adb0 service nova] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:12:29.048966 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cde8ca-96c0-4a60-887a-4abfe0f4adb0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:12:29.050695 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-81cde8ca-96c0-4a60-887a-4abfe0f4adb0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:12:29.186495 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cde8ca-96c0-4a60-887a-4abfe0f4adb0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:12:29.189634 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cde8ca-96c0-4a60-887a-4abfe0f4adb0 service nova] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:12:29.213122 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 666/1363] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:12:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 503 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 23 15:12:32.114759 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44adb3f-e256-418d-b2ca-e9999b5a8e73 None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:12:32.115603 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44adb3f-e256-418d-b2ca-e9999b5a8e73 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:12:32.115603 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44adb3f-e256-418d-b2ca-e9999b5a8e73 None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:12:32.121090 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 698/1364] 10.222.0.55 () {54 vars in 927 bytes} [Fri Sep 23 15:12:32 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 23 15:12:32.124244 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667c49dc-6ce4-4633-9999-dfa47fcfcd83 None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:12:32.126149 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667c49dc-6ce4-4633-9999-dfa47fcfcd83 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:12:32.127704 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667c49dc-6ce4-4633-9999-dfa47fcfcd83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:12:32.161345 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:12:32.267856 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-667c49dc-6ce4-4633-9999-dfa47fcfcd83 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:12:32.270028 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-667c49dc-6ce4-4633-9999-dfa47fcfcd83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:12:32.439619 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 667/1365] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Sep 23 15:12:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:12:32.716810 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6c358c-40ce-455d-97ef-e4f1c3d9f156 None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:12:32.717023 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6c358c-40ce-455d-97ef-e4f1c3d9f156 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:12:32.717257 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6c358c-40ce-455d-97ef-e4f1c3d9f156 None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:12:32.718655 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 699/1366] 10.222.0.55 () {54 vars in 927 bytes} [Fri Sep 23 15:12:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 23 15:12:32.724845 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33422303-d941-433b-958e-190ae3be1da2 None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:12:32.725150 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33422303-d941-433b-958e-190ae3be1da2 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:12:32.725658 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33422303-d941-433b-958e-190ae3be1da2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:12:32.758353 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ecb92a-7d11-4535-92f3-a97f8f21ba55 None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:12:32.758683 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ecb92a-7d11-4535-92f3-a97f8f21ba55 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:12:32.758902 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ecb92a-7d11-4535-92f3-a97f8f21ba55 None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:12:32.760287 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 700/1367] 10.222.0.55 () {54 vars in 961 bytes} [Fri Sep 23 15:12:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 23 15:12:32.766931 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7814ff98-aeb9-45ef-86f9-ea9dcbfb50bd None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:12:32.767558 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7814ff98-aeb9-45ef-86f9-ea9dcbfb50bd None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:12:32.774315 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7814ff98-aeb9-45ef-86f9-ea9dcbfb50bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:12:32.816700 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-33422303-d941-433b-958e-190ae3be1da2 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:12:32.818082 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-33422303-d941-433b-958e-190ae3be1da2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:12:32.876222 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:12:32.877847 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-7814ff98-aeb9-45ef-86f9-ea9dcbfb50bd None None] MFA Rules not processed for user `b3c6c7a1325a47be924268e1aa1a7a18`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:12:32.879530 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-7814ff98-aeb9-45ef-86f9-ea9dcbfb50bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:12:32.954520 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 668/1368] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Sep 23 15:12:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:12:33.023713 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:12:33.047291 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 701/1369] 10.222.0.55 () {58 vars in 1058 bytes} [Fri Sep 23 15:12:32 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 15:12:33.061111 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33422303-d941-433b-958e-190ae3be1da2 None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:12:33.061111 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-33422303-d941-433b-958e-190ae3be1da2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:12:33.195726 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33422303-d941-433b-958e-190ae3be1da2 None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:12:33.259402 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9a8383-bf16-437c-8997-6e7beeec7f37 None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:12:33.259933 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9a8383-bf16-437c-8997-6e7beeec7f37 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:12:33.259933 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9a8383-bf16-437c-8997-6e7beeec7f37 None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:12:33.261315 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 702/1370] 10.222.0.55 () {54 vars in 963 bytes} [Fri Sep 23 15:12:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 23 15:12:33.267072 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c14dfe7-ba31-4290-beaf-675ea4c3dd63 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:12:33.267930 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c14dfe7-ba31-4290-beaf-675ea4c3dd63 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:12:33.268368 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c14dfe7-ba31-4290-beaf-675ea4c3dd63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:12:33.269488 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:12:33.338356 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79892513-1e40-48b4-a5d9-31f0797d6e3b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3c6c7a1325a47be924268e1aa1a7a18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:12:33.339167 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79892513-1e40-48b4-a5d9-31f0797d6e3b service nova] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:12:33.339444 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79892513-1e40-48b4-a5d9-31f0797d6e3b service nova] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:12:33.339713 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79892513-1e40-48b4-a5d9-31f0797d6e3b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:12:33.341392 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-79892513-1e40-48b4-a5d9-31f0797d6e3b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:12:33.361984 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-7c14dfe7-ba31-4290-beaf-675ea4c3dd63 None None] MFA Rules not processed for user `feca6278be904ef8b0dc580a67e477f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:12:33.363650 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-7c14dfe7-ba31-4290-beaf-675ea4c3dd63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:12:33.446923 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79892513-1e40-48b4-a5d9-31f0797d6e3b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:12:33.448612 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79892513-1e40-48b4-a5d9-31f0797d6e3b service nova] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:12:33.463656 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 703/1371] 10.222.0.55 () {58 vars in 1060 bytes} [Fri Sep 23 15:12:33 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 15:12:33.464962 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c14dfe7-ba31-4290-beaf-675ea4c3dd63 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:12:33.474194 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 669/1372] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:12:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:12:33.474429 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c14dfe7-ba31-4290-beaf-675ea4c3dd63 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:12:33.568997 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:12:33.571557 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab67b07-9ca0-4053-8c10-13a9d2276f3b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feca6278be904ef8b0dc580a67e477f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:12:33.572161 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab67b07-9ca0-4053-8c10-13a9d2276f3b service cinder] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:12:33.582385 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab67b07-9ca0-4053-8c10-13a9d2276f3b service cinder] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:12:33.582385 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab67b07-9ca0-4053-8c10-13a9d2276f3b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:12:33.582385 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-aab67b07-9ca0-4053-8c10-13a9d2276f3b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:12:33.708691 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab67b07-9ca0-4053-8c10-13a9d2276f3b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:12:33.710333 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab67b07-9ca0-4053-8c10-13a9d2276f3b service cinder] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:12:33.735417 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 704/1373] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:12:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 271 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:12:36.125941 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffccf6d-091a-407a-8428-79ab04ecbdc7 None None] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:12:36.126772 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffccf6d-091a-407a-8428-79ab04ecbdc7 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:12:36.127024 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffccf6d-091a-407a-8428-79ab04ecbdc7 None None] PATH_INFO: `/` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:12:36.128620 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 670/1374] 10.222.0.55 () {54 vars in 927 bytes} [Fri Sep 23 15:12:36 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:12:36.134849 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d44e78-6e98-4880-8ec3-8e942dd315de None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:12:36.135320 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d44e78-6e98-4880-8ec3-8e942dd315de None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:12:36.135454 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d44e78-6e98-4880-8ec3-8e942dd315de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:12:36.221029 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-16d44e78-6e98-4880-8ec3-8e942dd315de None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:12:36.222284 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-16d44e78-6e98-4880-8ec3-8e942dd315de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:12:36.287163 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:12:36.356234 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 705/1375] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Sep 23 15:12:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:12:36.685430 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffccf6d-091a-407a-8428-79ab04ecbdc7 None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:12:36.687518 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-0ffccf6d-091a-407a-8428-79ab04ecbdc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:12:36.771701 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:12:36.869102 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffccf6d-091a-407a-8428-79ab04ecbdc7 None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:12:37.002484 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907e88c0-8fd9-42d6-8054-08ca410b77ef service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3c6c7a1325a47be924268e1aa1a7a18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:12:37.003093 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907e88c0-8fd9-42d6-8054-08ca410b77ef service nova] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:12:37.003370 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907e88c0-8fd9-42d6-8054-08ca410b77ef service nova] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:12:37.003638 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907e88c0-8fd9-42d6-8054-08ca410b77ef service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:12:37.005268 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-907e88c0-8fd9-42d6-8054-08ca410b77ef service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:12:37.037408 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:12:37.117326 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907e88c0-8fd9-42d6-8054-08ca410b77ef service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:12:37.119302 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907e88c0-8fd9-42d6-8054-08ca410b77ef service nova] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:12:37.145529 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 671/1376] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:12:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 461 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:12:39.564310 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fa8ae4-8182-4a35-85c8-939abc733304 None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:12:39.564310 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fa8ae4-8182-4a35-85c8-939abc733304 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:12:39.564780 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fa8ae4-8182-4a35-85c8-939abc733304 None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:12:39.570088 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 706/1377] 10.222.0.55 () {54 vars in 927 bytes} [Fri Sep 23 15:12:39 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:12:39.571772 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6514ccc8-1077-45ac-95d2-31c17c4cf3df None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:12:39.572274 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6514ccc8-1077-45ac-95d2-31c17c4cf3df None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:12:39.572645 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6514ccc8-1077-45ac-95d2-31c17c4cf3df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:12:39.647207 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-6514ccc8-1077-45ac-95d2-31c17c4cf3df None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:12:39.648468 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-6514ccc8-1077-45ac-95d2-31c17c4cf3df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:12:39.760093 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 672/1378] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Sep 23 15:12:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:12:40.051030 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fa8ae4-8182-4a35-85c8-939abc733304 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:12:40.053385 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-89fa8ae4-8182-4a35-85c8-939abc733304 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:12:40.232614 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fa8ae4-8182-4a35-85c8-939abc733304 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:12:40.275887 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:12:40.353758 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8556e2f5-032f-48eb-807a-39792c4e2291 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3c6c7a1325a47be924268e1aa1a7a18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:12:40.354377 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8556e2f5-032f-48eb-807a-39792c4e2291 service nova] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:12:40.354654 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8556e2f5-032f-48eb-807a-39792c4e2291 service nova] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:12:40.354924 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8556e2f5-032f-48eb-807a-39792c4e2291 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:12:40.356527 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-8556e2f5-032f-48eb-807a-39792c4e2291 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:12:40.451714 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8556e2f5-032f-48eb-807a-39792c4e2291 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:12:40.453943 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8556e2f5-032f-48eb-807a-39792c4e2291 service nova] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:12:40.472770 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 707/1379] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:12:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:12:43.173420 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95ab449-e9c7-49bb-8ff6-c24455007675 None None] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:12:43.173971 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95ab449-e9c7-49bb-8ff6-c24455007675 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:12:43.174128 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95ab449-e9c7-49bb-8ff6-c24455007675 None None] PATH_INFO: `/` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:12:43.175942 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 673/1380] 10.222.0.55 () {54 vars in 927 bytes} [Fri Sep 23 15:12:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:12:43.181591 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274d52ed-9e71-4243-9199-435cb4106435 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:12:43.182036 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274d52ed-9e71-4243-9199-435cb4106435 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:12:43.182130 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274d52ed-9e71-4243-9199-435cb4106435 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:12:43.238892 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:12:43.254510 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-274d52ed-9e71-4243-9199-435cb4106435 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:12:43.255722 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-274d52ed-9e71-4243-9199-435cb4106435 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:12:43.377275 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 708/1381] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Sep 23 15:12:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:12:43.672202 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95ab449-e9c7-49bb-8ff6-c24455007675 None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:12:43.673718 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-e95ab449-e9c7-49bb-8ff6-c24455007675 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:12:43.677626 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:12:43.840356 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95ab449-e9c7-49bb-8ff6-c24455007675 None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:12:43.926169 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:12:43.969728 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdc4ef9-eefb-4de7-b9d1-1754892d5ee1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3c6c7a1325a47be924268e1aa1a7a18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:12:43.970344 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdc4ef9-eefb-4de7-b9d1-1754892d5ee1 service nova] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:12:43.970622 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdc4ef9-eefb-4de7-b9d1-1754892d5ee1 service nova] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:12:43.972737 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdc4ef9-eefb-4de7-b9d1-1754892d5ee1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:12:43.972737 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-7cdc4ef9-eefb-4de7-b9d1-1754892d5ee1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:12:44.073673 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdc4ef9-eefb-4de7-b9d1-1754892d5ee1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:12:44.075760 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdc4ef9-eefb-4de7-b9d1-1754892d5ee1 service nova] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:12:44.097867 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 674/1382] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:12:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 23 15:12:46.537315 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83299e04-7022-419f-bbe2-78a9d24cd1ca None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:12:46.538047 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83299e04-7022-419f-bbe2-78a9d24cd1ca None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:12:46.538047 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83299e04-7022-419f-bbe2-78a9d24cd1ca None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:12:46.539532 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 709/1383] 10.222.0.55 () {54 vars in 927 bytes} [Fri Sep 23 15:12:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:12:46.545806 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e018fb5d-3f70-470f-81fd-bb5e7ec6ec10 None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:12:46.546860 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e018fb5d-3f70-470f-81fd-bb5e7ec6ec10 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:12:46.547414 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e018fb5d-3f70-470f-81fd-bb5e7ec6ec10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:12:46.633236 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-e018fb5d-3f70-470f-81fd-bb5e7ec6ec10 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:12:46.634796 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-e018fb5d-3f70-470f-81fd-bb5e7ec6ec10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:12:46.683541 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:12:46.745674 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 675/1384] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Sep 23 15:12:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:12:47.049348 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83299e04-7022-419f-bbe2-78a9d24cd1ca None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:12:47.051411 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-83299e04-7022-419f-bbe2-78a9d24cd1ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:12:47.202163 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:12:47.241199 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83299e04-7022-419f-bbe2-78a9d24cd1ca None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:12:47.385247 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e62628-e179-4513-9bfe-710c6e712b04 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3c6c7a1325a47be924268e1aa1a7a18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:12:47.385798 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e62628-e179-4513-9bfe-710c6e712b04 service nova] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:12:47.386083 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e62628-e179-4513-9bfe-710c6e712b04 service nova] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:12:47.386368 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e62628-e179-4513-9bfe-710c6e712b04 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:12:47.388055 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-e2e62628-e179-4513-9bfe-710c6e712b04 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:12:47.487313 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:12:47.502210 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e62628-e179-4513-9bfe-710c6e712b04 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:12:47.504165 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e62628-e179-4513-9bfe-710c6e712b04 service nova] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:12:47.524367 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 710/1385] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:12:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 476 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:12:49.645276 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e41f9ce-1537-4f77-a93e-ef8a95ba0c63 None None] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:12:49.645878 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e41f9ce-1537-4f77-a93e-ef8a95ba0c63 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:12:49.645878 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e41f9ce-1537-4f77-a93e-ef8a95ba0c63 None None] PATH_INFO: `/` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:12:49.647213 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 676/1386] 10.222.0.55 () {54 vars in 927 bytes} [Fri Sep 23 15:12:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:12:49.653381 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9979e9c1-2498-4194-8c4c-5c31babd9b0d None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:12:49.653934 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9979e9c1-2498-4194-8c4c-5c31babd9b0d None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:12:49.653934 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9979e9c1-2498-4194-8c4c-5c31babd9b0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:12:49.727851 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-9979e9c1-2498-4194-8c4c-5c31babd9b0d None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:12:49.729160 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-9979e9c1-2498-4194-8c4c-5c31babd9b0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:12:49.765419 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620c60b1-9dfd-4b93-87d7-9994adea9950 None None] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:12:49.765707 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620c60b1-9dfd-4b93-87d7-9994adea9950 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:12:49.765989 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620c60b1-9dfd-4b93-87d7-9994adea9950 None None] PATH_INFO: `/` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:12:49.767567 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 677/1387] 10.222.0.55 () {54 vars in 927 bytes} [Fri Sep 23 15:12:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:12:49.772764 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6799a3c3-2748-4aa8-8e09-addb403f9cc5 None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:12:49.773133 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6799a3c3-2748-4aa8-8e09-addb403f9cc5 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:12:49.773416 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6799a3c3-2748-4aa8-8e09-addb403f9cc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:12:49.834804 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 711/1388] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Sep 23 15:12:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:12:49.846240 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-6799a3c3-2748-4aa8-8e09-addb403f9cc5 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:12:49.847469 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-6799a3c3-2748-4aa8-8e09-addb403f9cc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:12:49.953711 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 678/1389] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Sep 23 15:12:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 15:12:50.140411 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d83bfe4-0451-4a5d-9e60-4296f50fb0ab None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:12:50.140857 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d83bfe4-0451-4a5d-9e60-4296f50fb0ab None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:12:50.141342 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d83bfe4-0451-4a5d-9e60-4296f50fb0ab None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:12:50.142984 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 712/1390] 10.222.0.55 () {54 vars in 962 bytes} [Fri Sep 23 15:12:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:12:50.148577 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f4dcd7-726c-4480-ac7f-c6a6531bc9dd None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:12:50.148881 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f4dcd7-726c-4480-ac7f-c6a6531bc9dd None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:12:50.149136 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f4dcd7-726c-4480-ac7f-c6a6531bc9dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:12:50.176945 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:12:50.230015 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-c1f4dcd7-726c-4480-ac7f-c6a6531bc9dd None None] MFA Rules not processed for user `51c39f1d05634589a88c0b6477bd014e`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:12:50.231308 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-c1f4dcd7-726c-4480-ac7f-c6a6531bc9dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:12:50.245314 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d83bfe4-0451-4a5d-9e60-4296f50fb0ab None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:12:50.246843 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-6d83bfe4-0451-4a5d-9e60-4296f50fb0ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:12:50.315770 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:12:50.318469 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 679/1391] 10.222.0.55 () {58 vars in 1059 bytes} [Fri Sep 23 15:12:50 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:12:50.328422 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f4dcd7-726c-4480-ac7f-c6a6531bc9dd None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:12:50.336785 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f4dcd7-726c-4480-ac7f-c6a6531bc9dd None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:12:50.404137 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d83bfe4-0451-4a5d-9e60-4296f50fb0ab None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:12:50.417863 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252a38ba-775d-4b92-8ea2-b4015781114f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51c39f1d05634589a88c0b6477bd014e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:12:50.418480 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252a38ba-775d-4b92-8ea2-b4015781114f service glance] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:12:50.418758 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252a38ba-775d-4b92-8ea2-b4015781114f service glance] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:12:50.419031 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252a38ba-775d-4b92-8ea2-b4015781114f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:12:50.420671 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-252a38ba-775d-4b92-8ea2-b4015781114f service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:12:50.472105 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:12:50.538034 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87fc268-dccc-4761-b612-155561582f17 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3c6c7a1325a47be924268e1aa1a7a18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:12:50.538862 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87fc268-dccc-4761-b612-155561582f17 service nova] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:12:50.539321 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87fc268-dccc-4761-b612-155561582f17 service nova] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:12:50.539694 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87fc268-dccc-4761-b612-155561582f17 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:12:50.541518 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-e87fc268-dccc-4761-b612-155561582f17 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:12:50.561193 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252a38ba-775d-4b92-8ea2-b4015781114f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:12:50.562643 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252a38ba-775d-4b92-8ea2-b4015781114f service glance] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:12:50.566305 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:12:50.584501 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 680/1392] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:12:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 257 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 23 15:12:50.638635 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87fc268-dccc-4761-b612-155561582f17 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:12:50.641535 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87fc268-dccc-4761-b612-155561582f17 service nova] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:12:50.662116 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 713/1393] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:12:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 23 15:12:51.484542 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d8cfe3-4e80-4b49-bbcd-d1f9db13d785 None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:12:51.485035 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d8cfe3-4e80-4b49-bbcd-d1f9db13d785 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:12:51.485439 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d8cfe3-4e80-4b49-bbcd-d1f9db13d785 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:12:51.589922 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-52d8cfe3-4e80-4b49-bbcd-d1f9db13d785 None None] MFA Rules not processed for user `68206a6b4f244d45bf7bf1504396e430`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:12:51.591058 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-52d8cfe3-4e80-4b49-bbcd-d1f9db13d785 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:12:51.691236 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 681/1394] 10.222.0.55 () {58 vars in 1009 bytes} [Fri Sep 23 15:12:51 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:12:51.736227 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c02ad0a-c427-4cf0-979e-c4e5d2d4b7b6 None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:12:51.736732 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c02ad0a-c427-4cf0-979e-c4e5d2d4b7b6 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:12:51.736923 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c02ad0a-c427-4cf0-979e-c4e5d2d4b7b6 None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:12:51.738402 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 714/1395] 10.222.0.55 () {54 vars in 943 bytes} [Fri Sep 23 15:12:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:12:51.745408 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c219476-c72d-46be-9d33-443f1842d611 None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:12:51.745699 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c219476-c72d-46be-9d33-443f1842d611 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:12:51.745998 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c219476-c72d-46be-9d33-443f1842d611 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:12:51.752457 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:12:51.836281 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-8c219476-c72d-46be-9d33-443f1842d611 None None] MFA Rules not processed for user `2228c0b49a0249bbaf524a8c4e9e108b`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:12:51.837529 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-8c219476-c72d-46be-9d33-443f1842d611 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:12:51.961447 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 682/1396] 10.222.0.55 () {58 vars in 1040 bytes} [Fri Sep 23 15:12:51 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:12:51.972591 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c02ad0a-c427-4cf0-979e-c4e5d2d4b7b6 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:12:51.974346 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-4c02ad0a-c427-4cf0-979e-c4e5d2d4b7b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:12:52.137959 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c02ad0a-c427-4cf0-979e-c4e5d2d4b7b6 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:12:52.154752 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:12:52.259081 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97140a51-9599-4223-b6a2-1141ff1dd972 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2228c0b49a0249bbaf524a8c4e9e108b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:12:52.259702 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97140a51-9599-4223-b6a2-1141ff1dd972 service swift] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:12:52.260002 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97140a51-9599-4223-b6a2-1141ff1dd972 service swift] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:12:52.260286 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97140a51-9599-4223-b6a2-1141ff1dd972 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:12:52.262004 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-97140a51-9599-4223-b6a2-1141ff1dd972 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:12:52.353723 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97140a51-9599-4223-b6a2-1141ff1dd972 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:12:52.355326 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97140a51-9599-4223-b6a2-1141ff1dd972 service swift] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:12:52.364129 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 715/1397] 10.222.0.55 () {60 vars in 1384 bytes} [Fri Sep 23 15:12:51 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 23 15:12:53.126093 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853730f0-fc72-47a1-968c-efbdc6de1721 None None] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:12:53.126645 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853730f0-fc72-47a1-968c-efbdc6de1721 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:12:53.126732 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853730f0-fc72-47a1-968c-efbdc6de1721 None None] PATH_INFO: `/` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:12:53.129780 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 683/1398] 10.222.0.55 () {54 vars in 927 bytes} [Fri Sep 23 15:12:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:12:53.133657 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe86cae-e5af-4c87-9acc-2d8f7f1c0ad8 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:12:53.134129 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe86cae-e5af-4c87-9acc-2d8f7f1c0ad8 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:12:53.134279 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe86cae-e5af-4c87-9acc-2d8f7f1c0ad8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:12:53.205096 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:12:53.214432 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-ebe86cae-e5af-4c87-9acc-2d8f7f1c0ad8 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:12:53.215899 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-ebe86cae-e5af-4c87-9acc-2d8f7f1c0ad8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:12:53.330617 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 716/1399] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Sep 23 15:12:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:12:53.639783 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853730f0-fc72-47a1-968c-efbdc6de1721 None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:12:53.642435 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-853730f0-fc72-47a1-968c-efbdc6de1721 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:12:53.739193 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:12:53.834163 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853730f0-fc72-47a1-968c-efbdc6de1721 None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:12:53.984414 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d92f95a-07a7-48c8-ae28-f7b0576cb27d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3c6c7a1325a47be924268e1aa1a7a18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:12:53.985708 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d92f95a-07a7-48c8-ae28-f7b0576cb27d service nova] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:12:53.985984 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d92f95a-07a7-48c8-ae28-f7b0576cb27d service nova] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:12:53.986280 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d92f95a-07a7-48c8-ae28-f7b0576cb27d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:12:53.987965 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-9d92f95a-07a7-48c8-ae28-f7b0576cb27d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:12:54.022095 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:12:54.098799 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d92f95a-07a7-48c8-ae28-f7b0576cb27d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:12:54.100555 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d92f95a-07a7-48c8-ae28-f7b0576cb27d service nova] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:12:54.121281 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 684/1400] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:12:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 482 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:12:57.372768 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec311816-640a-4ede-b09c-a162c1024f76 None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:12:57.373378 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec311816-640a-4ede-b09c-a162c1024f76 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:12:57.373378 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec311816-640a-4ede-b09c-a162c1024f76 None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:12:57.375594 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 717/1401] 10.222.0.55 () {54 vars in 927 bytes} [Fri Sep 23 15:12:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:12:57.384169 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e17465-6b03-4340-be4a-1bc4f817f5f1 None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:12:57.384732 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e17465-6b03-4340-be4a-1bc4f817f5f1 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:12:57.384979 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e17465-6b03-4340-be4a-1bc4f817f5f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:12:57.470660 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-68e17465-6b03-4340-be4a-1bc4f817f5f1 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:12:57.472012 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-68e17465-6b03-4340-be4a-1bc4f817f5f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:12:57.604025 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 685/1402] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Sep 23 15:12:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:12:57.900527 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec311816-640a-4ede-b09c-a162c1024f76 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:12:57.902078 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-ec311816-640a-4ede-b09c-a162c1024f76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:12:58.050891 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:12:58.091108 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec311816-640a-4ede-b09c-a162c1024f76 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:12:58.235647 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c185618a-8117-4e15-b1a8-a8b5a3ec0fab service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3c6c7a1325a47be924268e1aa1a7a18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:12:58.237444 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c185618a-8117-4e15-b1a8-a8b5a3ec0fab service nova] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:12:58.237444 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c185618a-8117-4e15-b1a8-a8b5a3ec0fab service nova] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:12:58.237444 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c185618a-8117-4e15-b1a8-a8b5a3ec0fab service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:12:58.239421 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-c185618a-8117-4e15-b1a8-a8b5a3ec0fab service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:12:58.358659 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:12:58.363500 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c185618a-8117-4e15-b1a8-a8b5a3ec0fab service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:12:58.365290 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c185618a-8117-4e15-b1a8-a8b5a3ec0fab service nova] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:12:58.386953 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 718/1403] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:12:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 487 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:13:00.845846 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e66cfec-2d97-4693-8d99-2b2e51498f91 None None] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:13:00.846422 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e66cfec-2d97-4693-8d99-2b2e51498f91 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:13:00.846422 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e66cfec-2d97-4693-8d99-2b2e51498f91 None None] PATH_INFO: `/` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:13:00.848043 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 686/1404] 10.222.0.55 () {54 vars in 927 bytes} [Fri Sep 23 15:13:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:13:00.853628 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98ae421-5c64-46f5-b3a7-fecd1a82b24a None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:13:00.854169 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98ae421-5c64-46f5-b3a7-fecd1a82b24a None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:13:00.854169 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98ae421-5c64-46f5-b3a7-fecd1a82b24a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:13:00.935893 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-e98ae421-5c64-46f5-b3a7-fecd1a82b24a None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:13:00.937946 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-e98ae421-5c64-46f5-b3a7-fecd1a82b24a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:13:01.114373 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 719/1405] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Sep 23 15:13:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:13:01.432990 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e66cfec-2d97-4693-8d99-2b2e51498f91 None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:13:01.434554 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-7e66cfec-2d97-4693-8d99-2b2e51498f91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:13:01.440360 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:13:01.618152 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e66cfec-2d97-4693-8d99-2b2e51498f91 None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:13:01.708387 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:13:01.758609 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3394b4d-f8cf-4058-bc19-34d4581a1a20 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3c6c7a1325a47be924268e1aa1a7a18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:13:01.759226 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3394b4d-f8cf-4058-bc19-34d4581a1a20 service nova] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:13:01.759507 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3394b4d-f8cf-4058-bc19-34d4581a1a20 service nova] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:13:01.759778 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3394b4d-f8cf-4058-bc19-34d4581a1a20 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:13:01.764317 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-a3394b4d-f8cf-4058-bc19-34d4581a1a20 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:13:01.867915 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3394b4d-f8cf-4058-bc19-34d4581a1a20 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:13:01.869756 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3394b4d-f8cf-4058-bc19-34d4581a1a20 service nova] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:13:01.893035 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 687/1406] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:13:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 461 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:13:04.336366 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11820b0a-cf85-475f-8528-a0f5eb196f68 None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:13:04.337502 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11820b0a-cf85-475f-8528-a0f5eb196f68 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:13:04.337782 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11820b0a-cf85-475f-8528-a0f5eb196f68 None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:13:04.339237 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 720/1407] 10.222.0.55 () {54 vars in 927 bytes} [Fri Sep 23 15:13:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:13:04.345581 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcedafc9-db5f-4874-999d-65cfde4481dd None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:13:04.346067 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcedafc9-db5f-4874-999d-65cfde4481dd None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:13:04.346169 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcedafc9-db5f-4874-999d-65cfde4481dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:13:04.430840 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-fcedafc9-db5f-4874-999d-65cfde4481dd None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:13:04.432128 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-fcedafc9-db5f-4874-999d-65cfde4481dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:13:04.479583 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:13:04.553888 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 688/1408] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Sep 23 15:13:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:13:04.850947 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11820b0a-cf85-475f-8528-a0f5eb196f68 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:13:04.852566 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-11820b0a-cf85-475f-8528-a0f5eb196f68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:13:04.937413 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:13:05.065890 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11820b0a-cf85-475f-8528-a0f5eb196f68 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:13:05.209005 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42afbbeb-e488-44b3-9613-afb250a1a8e3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3c6c7a1325a47be924268e1aa1a7a18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:13:05.209586 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42afbbeb-e488-44b3-9613-afb250a1a8e3 service nova] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:13:05.219834 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42afbbeb-e488-44b3-9613-afb250a1a8e3 service nova] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:13:05.219834 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42afbbeb-e488-44b3-9613-afb250a1a8e3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:13:05.219834 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-42afbbeb-e488-44b3-9613-afb250a1a8e3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:13:05.244862 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:13:05.330257 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42afbbeb-e488-44b3-9613-afb250a1a8e3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:13:05.332499 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42afbbeb-e488-44b3-9613-afb250a1a8e3 service nova] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:13:05.356689 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 721/1409] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:13:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 506 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:13:14.552023 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=26 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:13:15.353466 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=34 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:13:17.620677 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b8f65e-b85a-45d6-9839-464b63d3e3cb None None] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:13:17.621197 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b8f65e-b85a-45d6-9839-464b63d3e3cb None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:13:17.621197 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b8f65e-b85a-45d6-9839-464b63d3e3cb None None] PATH_INFO: `/` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:13:17.622573 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 689/1410] 10.222.0.55 () {54 vars in 918 bytes} [Fri Sep 23 15:13:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:13:17.628831 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f3e282-6838-4bb4-9813-9f331677f416 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:13:17.629278 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f3e282-6838-4bb4-9813-9f331677f416 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:13:17.629378 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f3e282-6838-4bb4-9813-9f331677f416 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:13:17.709514 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-87f3e282-6838-4bb4-9813-9f331677f416 None None] MFA Rules not processed for user `09f1a55d180642c781a6e0b017d9f0aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:13:17.710717 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-87f3e282-6838-4bb4-9813-9f331677f416 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:13:17.818665 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 722/1411] 10.222.0.55 () {58 vars in 1015 bytes} [Fri Sep 23 15:13:17 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:13:17.834034 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87300a0-def0-4075-8a9b-03dfb188581f None None] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:13:17.834320 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87300a0-def0-4075-8a9b-03dfb188581f None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:13:17.834597 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87300a0-def0-4075-8a9b-03dfb188581f None None] PATH_INFO: `/` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:13:17.835914 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 690/1412] 10.222.0.55 () {54 vars in 918 bytes} [Fri Sep 23 15:13:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:13:17.840961 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef6b846-1b6f-4ed7-a8d2-9752b6dd0c5d None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:13:17.841251 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef6b846-1b6f-4ed7-a8d2-9752b6dd0c5d None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:13:17.841525 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef6b846-1b6f-4ed7-a8d2-9752b6dd0c5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:13:17.918708 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:13:17.922172 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-8ef6b846-1b6f-4ed7-a8d2-9752b6dd0c5d None None] MFA Rules not processed for user `b3c6c7a1325a47be924268e1aa1a7a18`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:13:17.923402 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-8ef6b846-1b6f-4ed7-a8d2-9752b6dd0c5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:13:18.042653 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 723/1413] 10.222.0.55 () {58 vars in 1015 bytes} [Fri Sep 23 15:13:17 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:13:20.696542 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46720933-7bc8-4e5d-be17-ad58676b5e33 None None] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:13:20.697113 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46720933-7bc8-4e5d-be17-ad58676b5e33 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:13:20.697113 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46720933-7bc8-4e5d-be17-ad58676b5e33 None None] PATH_INFO: `/` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:13:20.698430 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 691/1414] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 23 15:13:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:13:20.704579 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3264ac2-a7e1-4e2a-b6a1-4fb6082d37be None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:13:20.705053 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3264ac2-a7e1-4e2a-b6a1-4fb6082d37be None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:13:20.705153 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3264ac2-a7e1-4e2a-b6a1-4fb6082d37be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:13:20.771966 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-d3264ac2-a7e1-4e2a-b6a1-4fb6082d37be None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:13:20.773244 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-d3264ac2-a7e1-4e2a-b6a1-4fb6082d37be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:13:20.828174 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:13:20.869047 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 724/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 23 15:13:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:13:21.157642 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46720933-7bc8-4e5d-be17-ad58676b5e33 None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:13:21.159164 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-46720933-7bc8-4e5d-be17-ad58676b5e33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:13:21.315856 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46720933-7bc8-4e5d-be17-ad58676b5e33 None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:13:21.389502 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:13:21.431991 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70dd36ff-ef94-406c-8c30-cfa96040a701 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3c6c7a1325a47be924268e1aa1a7a18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:13:21.432623 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dd36ff-ef94-406c-8c30-cfa96040a701 service nova] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:13:21.432904 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dd36ff-ef94-406c-8c30-cfa96040a701 service nova] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:13:21.433170 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dd36ff-ef94-406c-8c30-cfa96040a701 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:13:21.434760 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-70dd36ff-ef94-406c-8c30-cfa96040a701 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:13:21.525038 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70dd36ff-ef94-406c-8c30-cfa96040a701 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:13:21.526631 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70dd36ff-ef94-406c-8c30-cfa96040a701 service nova] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:13:21.545225 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 692/1416] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:13:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:13:22.709183 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf212b40-0489-4083-b146-1c7473c7b770 None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:13:22.709753 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf212b40-0489-4083-b146-1c7473c7b770 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:13:22.709753 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf212b40-0489-4083-b146-1c7473c7b770 None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:13:22.711122 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 725/1417] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 23 15:13:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:13:22.718615 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c235f907-4fac-4853-8ac1-edd20f8b9a46 None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:13:22.719941 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c235f907-4fac-4853-8ac1-edd20f8b9a46 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:13:22.719941 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c235f907-4fac-4853-8ac1-edd20f8b9a46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:13:22.799754 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-c235f907-4fac-4853-8ac1-edd20f8b9a46 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:13:22.801018 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-c235f907-4fac-4853-8ac1-edd20f8b9a46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:13:22.829884 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:13:22.900752 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 693/1418] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 23 15:13:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:13:23.062556 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf212b40-0489-4083-b146-1c7473c7b770 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:13:23.064215 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-bf212b40-0489-4083-b146-1c7473c7b770 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:13:23.213161 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf212b40-0489-4083-b146-1c7473c7b770 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:13:23.254574 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:13:23.328089 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3271b0-3eb5-4a4a-8637-9a3689e83aa7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3c6c7a1325a47be924268e1aa1a7a18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:13:23.328600 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3271b0-3eb5-4a4a-8637-9a3689e83aa7 service nova] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:13:23.328881 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3271b0-3eb5-4a4a-8637-9a3689e83aa7 service nova] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:13:23.329155 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3271b0-3eb5-4a4a-8637-9a3689e83aa7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:13:23.330741 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-7d3271b0-3eb5-4a4a-8637-9a3689e83aa7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:13:23.419798 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3271b0-3eb5-4a4a-8637-9a3689e83aa7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:13:23.422003 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3271b0-3eb5-4a4a-8637-9a3689e83aa7 service nova] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:13:23.440232 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 726/1419] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:13:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:13:24.614610 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67785390-0693-4f28-aa74-e01ccb43ebb8 None None] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:13:24.615184 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67785390-0693-4f28-aa74-e01ccb43ebb8 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:13:24.615184 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67785390-0693-4f28-aa74-e01ccb43ebb8 None None] PATH_INFO: `/` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:13:24.616535 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 694/1420] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 23 15:13:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:13:24.622811 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181101a5-2fcb-4306-98c0-86154d893d98 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:13:24.623268 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181101a5-2fcb-4306-98c0-86154d893d98 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:13:24.623375 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181101a5-2fcb-4306-98c0-86154d893d98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:13:24.680481 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:13:24.702724 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-181101a5-2fcb-4306-98c0-86154d893d98 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:13:24.703963 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-181101a5-2fcb-4306-98c0-86154d893d98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:13:24.806252 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 727/1421] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 23 15:13:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:13:24.957971 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67785390-0693-4f28-aa74-e01ccb43ebb8 None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:13:24.959422 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-67785390-0693-4f28-aa74-e01ccb43ebb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:13:25.108652 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67785390-0693-4f28-aa74-e01ccb43ebb8 None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:13:25.120931 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:13:25.226872 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e128fd78-3a9e-42ce-a087-cf384e30d5c5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3c6c7a1325a47be924268e1aa1a7a18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:13:25.227497 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e128fd78-3a9e-42ce-a087-cf384e30d5c5 service nova] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:13:25.227778 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e128fd78-3a9e-42ce-a087-cf384e30d5c5 service nova] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:13:25.228084 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e128fd78-3a9e-42ce-a087-cf384e30d5c5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:13:25.229790 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-e128fd78-3a9e-42ce-a087-cf384e30d5c5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:13:25.324329 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e128fd78-3a9e-42ce-a087-cf384e30d5c5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:13:25.325958 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e128fd78-3a9e-42ce-a087-cf384e30d5c5 service nova] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:13:25.345490 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 695/1422] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:13:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:13:26.467688 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea516a72-f16c-4296-8df4-098718186e91 None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:13:26.467688 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea516a72-f16c-4296-8df4-098718186e91 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:13:26.467688 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea516a72-f16c-4296-8df4-098718186e91 None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:13:26.467688 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 728/1423] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 23 15:13:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:13:26.476643 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0759b437-d36d-4b54-8353-b7cf913cc5d6 None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:13:26.477138 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0759b437-d36d-4b54-8353-b7cf913cc5d6 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:13:26.477239 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0759b437-d36d-4b54-8353-b7cf913cc5d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:13:26.501070 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:13:26.555639 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-0759b437-d36d-4b54-8353-b7cf913cc5d6 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:13:26.556875 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-0759b437-d36d-4b54-8353-b7cf913cc5d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:13:26.652628 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 696/1424] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 23 15:13:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:13:26.804044 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea516a72-f16c-4296-8df4-098718186e91 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:13:26.805553 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-ea516a72-f16c-4296-8df4-098718186e91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:13:26.917691 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:13:26.954894 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea516a72-f16c-4296-8df4-098718186e91 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:13:27.068056 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b361129-6109-4a29-9f1e-4cd16eac8c3f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3c6c7a1325a47be924268e1aa1a7a18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:13:27.068577 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b361129-6109-4a29-9f1e-4cd16eac8c3f service nova] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:13:27.068854 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b361129-6109-4a29-9f1e-4cd16eac8c3f service nova] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:13:27.069125 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b361129-6109-4a29-9f1e-4cd16eac8c3f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:13:27.070703 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-5b361129-6109-4a29-9f1e-4cd16eac8c3f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:13:27.149287 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:13:27.163621 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b361129-6109-4a29-9f1e-4cd16eac8c3f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:13:27.165290 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b361129-6109-4a29-9f1e-4cd16eac8c3f service nova] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:13:27.183320 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 729/1425] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:13:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:13:28.306649 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30efd60b-d834-487c-9831-6d98edc34653 None None] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:13:28.307240 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30efd60b-d834-487c-9831-6d98edc34653 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:13:28.307240 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30efd60b-d834-487c-9831-6d98edc34653 None None] PATH_INFO: `/` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:13:28.308546 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 697/1426] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 23 15:13:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:13:28.315044 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79856c5-d8e3-4ae5-adf0-6e14b37265a5 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:13:28.315495 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79856c5-d8e3-4ae5-adf0-6e14b37265a5 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:13:28.315623 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79856c5-d8e3-4ae5-adf0-6e14b37265a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:13:28.391879 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-d79856c5-d8e3-4ae5-adf0-6e14b37265a5 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:13:28.393309 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-d79856c5-d8e3-4ae5-adf0-6e14b37265a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:13:28.490129 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 730/1427] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 23 15:13:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:13:28.642570 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30efd60b-d834-487c-9831-6d98edc34653 None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:13:28.644042 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-30efd60b-d834-487c-9831-6d98edc34653 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:13:28.729175 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:13:28.793542 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30efd60b-d834-487c-9831-6d98edc34653 None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:13:28.910504 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77b14fe-c333-4198-b962-2c24d595ec4e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3c6c7a1325a47be924268e1aa1a7a18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:13:28.911042 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77b14fe-c333-4198-b962-2c24d595ec4e service nova] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:13:28.911322 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77b14fe-c333-4198-b962-2c24d595ec4e service nova] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:13:28.911594 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77b14fe-c333-4198-b962-2c24d595ec4e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:13:28.913236 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-a77b14fe-c333-4198-b962-2c24d595ec4e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:13:28.963530 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:13:29.005960 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77b14fe-c333-4198-b962-2c24d595ec4e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:13:29.007609 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77b14fe-c333-4198-b962-2c24d595ec4e service nova] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:13:29.026014 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 698/1428] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:13:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:13:30.206393 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ba6a6b-7373-4ceb-800c-ffbe5c197cd0 None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:13:30.210526 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ba6a6b-7373-4ceb-800c-ffbe5c197cd0 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:13:30.210526 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ba6a6b-7373-4ceb-800c-ffbe5c197cd0 None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:13:30.210526 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 731/1429] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 23 15:13:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:13:30.214647 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9634e61b-9956-4f0b-bfff-269b0b3c6fd3 None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:13:30.215058 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9634e61b-9956-4f0b-bfff-269b0b3c6fd3 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:13:30.215204 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9634e61b-9956-4f0b-bfff-269b0b3c6fd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:13:30.297050 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-9634e61b-9956-4f0b-bfff-269b0b3c6fd3 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:13:30.298317 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-9634e61b-9956-4f0b-bfff-269b0b3c6fd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:13:30.398030 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 699/1430] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 23 15:13:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:13:30.548874 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ba6a6b-7373-4ceb-800c-ffbe5c197cd0 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:13:30.550482 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-13ba6a6b-7373-4ceb-800c-ffbe5c197cd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:13:30.600284 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:13:30.700792 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ba6a6b-7373-4ceb-800c-ffbe5c197cd0 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:13:30.819435 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f1bf19-1803-4331-a7ac-b34854bb048e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3c6c7a1325a47be924268e1aa1a7a18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:13:30.820075 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f1bf19-1803-4331-a7ac-b34854bb048e service nova] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:13:30.820357 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f1bf19-1803-4331-a7ac-b34854bb048e service nova] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:13:30.820644 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f1bf19-1803-4331-a7ac-b34854bb048e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:13:30.822244 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-70f1bf19-1803-4331-a7ac-b34854bb048e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:13:30.830351 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:13:30.916692 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f1bf19-1803-4331-a7ac-b34854bb048e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:13:30.918281 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f1bf19-1803-4331-a7ac-b34854bb048e service nova] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:13:30.936622 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 732/1431] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:13:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:13:32.103407 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a747ec-d8c5-4aae-822e-563f9ab7f08c None None] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:13:32.103920 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a747ec-d8c5-4aae-822e-563f9ab7f08c None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:13:32.104113 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a747ec-d8c5-4aae-822e-563f9ab7f08c None None] PATH_INFO: `/` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:13:32.105456 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 700/1432] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 23 15:13:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:13:32.112356 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83e2f6e-e639-4734-a157-5f94fde2af8a None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:13:32.112803 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83e2f6e-e639-4734-a157-5f94fde2af8a None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:13:32.112923 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83e2f6e-e639-4734-a157-5f94fde2af8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:13:32.192733 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-b83e2f6e-e639-4734-a157-5f94fde2af8a None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:13:32.193863 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-b83e2f6e-e639-4734-a157-5f94fde2af8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:13:32.261337 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:13:32.289809 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 733/1433] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 23 15:13:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:13:32.446287 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a747ec-d8c5-4aae-822e-563f9ab7f08c None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:13:32.447706 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-22a747ec-d8c5-4aae-822e-563f9ab7f08c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:13:32.465288 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:13:32.592616 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a747ec-d8c5-4aae-822e-563f9ab7f08c None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:13:32.680748 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:13:32.707089 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43196ae9-aa6b-4023-bd5e-6d92a77ddd4a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3c6c7a1325a47be924268e1aa1a7a18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:13:32.707613 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43196ae9-aa6b-4023-bd5e-6d92a77ddd4a service nova] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:13:32.707895 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43196ae9-aa6b-4023-bd5e-6d92a77ddd4a service nova] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:13:32.708197 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43196ae9-aa6b-4023-bd5e-6d92a77ddd4a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:13:32.709792 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-43196ae9-aa6b-4023-bd5e-6d92a77ddd4a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:13:32.817801 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43196ae9-aa6b-4023-bd5e-6d92a77ddd4a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:13:32.819619 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43196ae9-aa6b-4023-bd5e-6d92a77ddd4a service nova] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:13:32.838608 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 701/1434] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:13:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:13:33.985428 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7378407b-bcd6-4528-b944-291a129cf7fb None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:13:33.985993 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7378407b-bcd6-4528-b944-291a129cf7fb None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:13:33.985993 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7378407b-bcd6-4528-b944-291a129cf7fb None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:13:33.987350 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 734/1435] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 23 15:13:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:13:33.993492 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b43780a-b77b-4284-877d-23868d134892 None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:13:33.993891 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b43780a-b77b-4284-877d-23868d134892 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:13:33.994036 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b43780a-b77b-4284-877d-23868d134892 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:13:34.072066 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-0b43780a-b77b-4284-877d-23868d134892 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:13:34.073276 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-0b43780a-b77b-4284-877d-23868d134892 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:13:34.115755 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:13:34.171298 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 702/1436] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 23 15:13:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:13:34.323468 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7378407b-bcd6-4528-b944-291a129cf7fb None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:13:34.325032 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-7378407b-bcd6-4528-b944-291a129cf7fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:13:34.473358 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7378407b-bcd6-4528-b944-291a129cf7fb None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:13:34.531810 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:13:34.590984 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70229cd5-b4b0-405b-bb00-67669d8886c2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3c6c7a1325a47be924268e1aa1a7a18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:13:34.591515 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70229cd5-b4b0-405b-bb00-67669d8886c2 service nova] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:13:34.591797 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70229cd5-b4b0-405b-bb00-67669d8886c2 service nova] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:13:34.592103 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70229cd5-b4b0-405b-bb00-67669d8886c2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:13:34.594854 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-70229cd5-b4b0-405b-bb00-67669d8886c2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:13:34.689988 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70229cd5-b4b0-405b-bb00-67669d8886c2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:13:34.691594 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70229cd5-b4b0-405b-bb00-67669d8886c2 service nova] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:13:34.711968 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 735/1437] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:13:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:13:35.838061 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9b384b-f38e-486d-bfd3-2f867e662a50 None None] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:13:35.838747 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9b384b-f38e-486d-bfd3-2f867e662a50 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:13:35.838747 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9b384b-f38e-486d-bfd3-2f867e662a50 None None] PATH_INFO: `/` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:13:35.839936 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 703/1438] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 23 15:13:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:13:35.846676 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883cd5a6-9fa5-44c7-88b2-e765e058e421 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:13:35.847380 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883cd5a6-9fa5-44c7-88b2-e765e058e421 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:13:35.847380 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883cd5a6-9fa5-44c7-88b2-e765e058e421 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:13:35.924723 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-883cd5a6-9fa5-44c7-88b2-e765e058e421 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:13:35.925847 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-883cd5a6-9fa5-44c7-88b2-e765e058e421 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:13:35.927908 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:13:36.024570 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 736/1439] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 23 15:13:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:13:36.175285 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9b384b-f38e-486d-bfd3-2f867e662a50 None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:13:36.176755 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-dc9b384b-f38e-486d-bfd3-2f867e662a50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:13:36.326184 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9b384b-f38e-486d-bfd3-2f867e662a50 None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:13:36.354006 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:13:36.439333 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4634036b-5318-4c43-b746-1e6eab946f47 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3c6c7a1325a47be924268e1aa1a7a18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:13:36.439884 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4634036b-5318-4c43-b746-1e6eab946f47 service nova] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:13:36.440186 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4634036b-5318-4c43-b746-1e6eab946f47 service nova] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:13:36.440458 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4634036b-5318-4c43-b746-1e6eab946f47 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:13:36.442045 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-4634036b-5318-4c43-b746-1e6eab946f47 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:13:36.533929 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4634036b-5318-4c43-b746-1e6eab946f47 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:13:36.535511 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4634036b-5318-4c43-b746-1e6eab946f47 service nova] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:13:36.554297 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 704/1440] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:13:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:13:38.623186 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252f8c7c-89f9-4b8e-9a16-78d891749685 None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:13:38.623822 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252f8c7c-89f9-4b8e-9a16-78d891749685 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:13:38.623822 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252f8c7c-89f9-4b8e-9a16-78d891749685 None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:13:38.625803 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 737/1441] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 23 15:13:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:13:38.631397 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fff7980-c310-499a-8df2-d92a53bf6349 None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:13:38.631844 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fff7980-c310-499a-8df2-d92a53bf6349 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:13:38.631949 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fff7980-c310-499a-8df2-d92a53bf6349 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:13:38.673698 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:13:38.698651 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-6fff7980-c310-499a-8df2-d92a53bf6349 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:13:38.699878 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-6fff7980-c310-499a-8df2-d92a53bf6349 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:13:38.800119 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 705/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 23 15:13:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:13:39.086276 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252f8c7c-89f9-4b8e-9a16-78d891749685 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:13:39.087895 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-252f8c7c-89f9-4b8e-9a16-78d891749685 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:13:39.193844 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252f8c7c-89f9-4b8e-9a16-78d891749685 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:13:39.227359 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:13:39.271021 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58512a80-b748-4e5f-be24-b3d8316d298c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd259178570864ffeb936ee10dfec6b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:13:39.271638 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58512a80-b748-4e5f-be24-b3d8316d298c service neutron] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:13:39.271969 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58512a80-b748-4e5f-be24-b3d8316d298c service neutron] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:13:39.272286 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58512a80-b748-4e5f-be24-b3d8316d298c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:13:39.273964 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-58512a80-b748-4e5f-be24-b3d8316d298c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:13:39.386570 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58512a80-b748-4e5f-be24-b3d8316d298c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:13:39.387922 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58512a80-b748-4e5f-be24-b3d8316d298c service neutron] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:13:39.407247 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 738/1443] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:13:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:13:41.729106 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1c1786-0b60-406e-9854-79e8f30be82a None None] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:13:41.729676 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1c1786-0b60-406e-9854-79e8f30be82a None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:13:41.729676 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1c1786-0b60-406e-9854-79e8f30be82a None None] PATH_INFO: `/` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:13:41.730996 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 706/1444] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 23 15:13:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:13:41.736928 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1894368a-026c-4e19-9b31-75279e508aec None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:13:41.737337 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1894368a-026c-4e19-9b31-75279e508aec None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:13:41.737469 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1894368a-026c-4e19-9b31-75279e508aec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:13:41.807800 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-1894368a-026c-4e19-9b31-75279e508aec None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:13:41.809036 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-1894368a-026c-4e19-9b31-75279e508aec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:13:41.822160 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:13:41.908513 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 739/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 23 15:13:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:13:42.204590 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1c1786-0b60-406e-9854-79e8f30be82a None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:13:42.206196 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-be1c1786-0b60-406e-9854-79e8f30be82a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:13:42.312293 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1c1786-0b60-406e-9854-79e8f30be82a None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:13:42.325390 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:13:42.386811 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7604474f-ae8f-453f-914d-69c04f1150e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd259178570864ffeb936ee10dfec6b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:13:42.387420 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7604474f-ae8f-453f-914d-69c04f1150e6 service neutron] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:13:42.387695 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7604474f-ae8f-453f-914d-69c04f1150e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:13:42.387963 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7604474f-ae8f-453f-914d-69c04f1150e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:13:42.390252 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-7604474f-ae8f-453f-914d-69c04f1150e6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:13:42.500293 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7604474f-ae8f-453f-914d-69c04f1150e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:13:42.501732 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7604474f-ae8f-453f-914d-69c04f1150e6 service neutron] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:13:42.520277 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 707/1446] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:13:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:13:45.166787 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae04c6b-c288-4096-ac8a-73fbb6451382 None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:13:45.167478 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae04c6b-c288-4096-ac8a-73fbb6451382 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:13:45.167478 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae04c6b-c288-4096-ac8a-73fbb6451382 None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:13:45.168710 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 740/1447] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 23 15:13:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 23 15:13:45.174904 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f3dcb6-c2bc-46ee-8a96-70d4420ef617 None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:13:45.175351 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f3dcb6-c2bc-46ee-8a96-70d4420ef617 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:13:45.175466 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f3dcb6-c2bc-46ee-8a96-70d4420ef617 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:13:45.225709 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:13:45.244011 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-73f3dcb6-c2bc-46ee-8a96-70d4420ef617 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:13:45.245243 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-73f3dcb6-c2bc-46ee-8a96-70d4420ef617 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:13:45.343733 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 708/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 23 15:13:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:13:45.631342 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae04c6b-c288-4096-ac8a-73fbb6451382 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:13:45.632958 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-aae04c6b-c288-4096-ac8a-73fbb6451382 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:13:45.736700 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae04c6b-c288-4096-ac8a-73fbb6451382 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:13:45.778044 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:13:45.812270 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4110c05c-fed2-4578-8fac-3c10dd9c6cfd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd259178570864ffeb936ee10dfec6b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:13:45.812886 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4110c05c-fed2-4578-8fac-3c10dd9c6cfd service neutron] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:13:45.813165 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4110c05c-fed2-4578-8fac-3c10dd9c6cfd service neutron] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:13:45.813435 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4110c05c-fed2-4578-8fac-3c10dd9c6cfd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:13:45.815025 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-4110c05c-fed2-4578-8fac-3c10dd9c6cfd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:13:46.045878 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4110c05c-fed2-4578-8fac-3c10dd9c6cfd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:13:46.047279 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4110c05c-fed2-4578-8fac-3c10dd9c6cfd service neutron] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:13:46.066374 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 741/1449] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:13:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:13:48.373033 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031b3832-81f8-49f0-987e-85bc1ebe32f2 None None] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:13:48.373575 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031b3832-81f8-49f0-987e-85bc1ebe32f2 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:13:48.373575 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031b3832-81f8-49f0-987e-85bc1ebe32f2 None None] PATH_INFO: `/` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:13:48.375558 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 709/1450] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 23 15:13:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 23 15:13:48.381170 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba99f98c-83ef-4425-9f05-30f3697e7a9d None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:13:48.381862 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba99f98c-83ef-4425-9f05-30f3697e7a9d None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:13:48.381862 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba99f98c-83ef-4425-9f05-30f3697e7a9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:13:48.448677 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-ba99f98c-83ef-4425-9f05-30f3697e7a9d None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:13:48.449898 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-ba99f98c-83ef-4425-9f05-30f3697e7a9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:13:48.471839 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:13:48.550080 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 742/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 23 15:13:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:13:48.838131 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031b3832-81f8-49f0-987e-85bc1ebe32f2 None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:13:48.839670 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-031b3832-81f8-49f0-987e-85bc1ebe32f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:13:48.944893 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031b3832-81f8-49f0-987e-85bc1ebe32f2 None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:13:48.966221 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:13:49.022235 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c662c8-008c-41e6-bb23-2b43884fa75f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd259178570864ffeb936ee10dfec6b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:13:49.022848 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c662c8-008c-41e6-bb23-2b43884fa75f service neutron] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:13:49.023127 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c662c8-008c-41e6-bb23-2b43884fa75f service neutron] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:13:49.023397 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c662c8-008c-41e6-bb23-2b43884fa75f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:13:49.025019 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-62c662c8-008c-41e6-bb23-2b43884fa75f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:13:49.150159 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c662c8-008c-41e6-bb23-2b43884fa75f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:13:49.151616 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c662c8-008c-41e6-bb23-2b43884fa75f service neutron] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:13:49.170213 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 710/1452] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:13:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:13:51.484009 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91d83cb-76c7-4bf3-81ec-afc4192c737c None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:13:51.485028 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91d83cb-76c7-4bf3-81ec-afc4192c737c None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:13:51.485028 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91d83cb-76c7-4bf3-81ec-afc4192c737c None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:13:51.486022 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 743/1453] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 23 15:13:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:13:51.491550 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9217466-39ac-4ddd-be5f-46d038644849 None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:13:51.492031 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9217466-39ac-4ddd-be5f-46d038644849 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:13:51.492156 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9217466-39ac-4ddd-be5f-46d038644849 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:13:51.558248 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:13:51.561053 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-e9217466-39ac-4ddd-be5f-46d038644849 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:13:51.562989 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-e9217466-39ac-4ddd-be5f-46d038644849 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:13:51.669812 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 711/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 23 15:13:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:13:51.966956 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91d83cb-76c7-4bf3-81ec-afc4192c737c None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:13:51.969145 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-a91d83cb-76c7-4bf3-81ec-afc4192c737c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:13:52.074900 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91d83cb-76c7-4bf3-81ec-afc4192c737c None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:13:52.125422 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:13:52.151172 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfcc0ac-5f89-498c-854b-df0b9ebeefe4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd259178570864ffeb936ee10dfec6b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:13:52.151764 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfcc0ac-5f89-498c-854b-df0b9ebeefe4 service neutron] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:13:52.152081 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfcc0ac-5f89-498c-854b-df0b9ebeefe4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:13:52.152354 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfcc0ac-5f89-498c-854b-df0b9ebeefe4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:13:52.154019 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-6dfcc0ac-5f89-498c-854b-df0b9ebeefe4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:13:52.266158 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfcc0ac-5f89-498c-854b-df0b9ebeefe4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:13:52.267838 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfcc0ac-5f89-498c-854b-df0b9ebeefe4 service neutron] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:13:52.288651 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 744/1455] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:13:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:13:54.574207 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bef329c-d94a-4ad6-beac-0ac09fa9acc8 None None] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:13:54.574815 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bef329c-d94a-4ad6-beac-0ac09fa9acc8 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:13:54.574815 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bef329c-d94a-4ad6-beac-0ac09fa9acc8 None None] PATH_INFO: `/` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:13:54.576092 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 712/1456] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 23 15:13:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:13:54.581652 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dfa616-838e-4970-8e0e-3b9d21c23d58 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:13:54.582107 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dfa616-838e-4970-8e0e-3b9d21c23d58 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:13:54.582203 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dfa616-838e-4970-8e0e-3b9d21c23d58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:13:54.650395 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-d9dfa616-838e-4970-8e0e-3b9d21c23d58 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:13:54.651702 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-d9dfa616-838e-4970-8e0e-3b9d21c23d58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:13:54.685035 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:13:54.757660 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 745/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 23 15:13:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:13:55.044248 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bef329c-d94a-4ad6-beac-0ac09fa9acc8 None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:13:55.045873 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-1bef329c-d94a-4ad6-beac-0ac09fa9acc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:13:55.155512 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bef329c-d94a-4ad6-beac-0ac09fa9acc8 None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:13:55.185726 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:13:55.231910 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46b17ff-3eb2-472b-9846-48df7a095690 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd259178570864ffeb936ee10dfec6b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:13:55.232572 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46b17ff-3eb2-472b-9846-48df7a095690 service neutron] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:13:55.232857 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46b17ff-3eb2-472b-9846-48df7a095690 service neutron] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:13:55.233188 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46b17ff-3eb2-472b-9846-48df7a095690 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:13:55.235449 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-e46b17ff-3eb2-472b-9846-48df7a095690 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:13:55.353373 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46b17ff-3eb2-472b-9846-48df7a095690 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:13:55.354731 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46b17ff-3eb2-472b-9846-48df7a095690 service neutron] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:13:55.374160 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 713/1458] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:13:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:13:57.643910 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b79333a-421a-41ce-a1ef-1784e8a9f54a None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:13:57.644515 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b79333a-421a-41ce-a1ef-1784e8a9f54a None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:13:57.644515 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b79333a-421a-41ce-a1ef-1784e8a9f54a None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:13:57.645895 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 746/1459] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 23 15:13:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:13:57.651459 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559de2aa-6ec5-4ea0-947f-0cec5423ce14 None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:13:57.651926 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559de2aa-6ec5-4ea0-947f-0cec5423ce14 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:13:57.652068 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559de2aa-6ec5-4ea0-947f-0cec5423ce14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:13:57.723285 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-559de2aa-6ec5-4ea0-947f-0cec5423ce14 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:13:57.724556 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-559de2aa-6ec5-4ea0-947f-0cec5423ce14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:13:57.733403 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:13:57.826584 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 714/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 23 15:13:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:13:58.113590 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b79333a-421a-41ce-a1ef-1784e8a9f54a None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:13:58.115201 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-8b79333a-421a-41ce-a1ef-1784e8a9f54a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:13:58.234361 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b79333a-421a-41ce-a1ef-1784e8a9f54a None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:13:58.293296 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:13:58.315128 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28703d91-6b52-4897-9092-987d2fa436fb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd259178570864ffeb936ee10dfec6b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:13:58.315752 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28703d91-6b52-4897-9092-987d2fa436fb service neutron] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:13:58.316078 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28703d91-6b52-4897-9092-987d2fa436fb service neutron] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:13:58.316466 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28703d91-6b52-4897-9092-987d2fa436fb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:13:58.318093 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-28703d91-6b52-4897-9092-987d2fa436fb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:13:58.449896 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28703d91-6b52-4897-9092-987d2fa436fb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:13:58.451329 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28703d91-6b52-4897-9092-987d2fa436fb service neutron] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:13:58.471264 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 747/1461] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:13:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:14:00.720148 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7f7e79-d04b-4e14-9a58-be0cc01fe948 None None] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:14:00.721011 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7f7e79-d04b-4e14-9a58-be0cc01fe948 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:14:00.721178 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7f7e79-d04b-4e14-9a58-be0cc01fe948 None None] PATH_INFO: `/` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:14:00.722551 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 715/1462] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 23 15:14:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:14:00.727793 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7e0eb3-9183-4f1c-8aeb-610f3a38632b None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:14:00.728306 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7e0eb3-9183-4f1c-8aeb-610f3a38632b None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:14:00.728402 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7e0eb3-9183-4f1c-8aeb-610f3a38632b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:14:00.796444 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-1d7e0eb3-9183-4f1c-8aeb-610f3a38632b None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:14:00.797672 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-1d7e0eb3-9183-4f1c-8aeb-610f3a38632b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:14:00.837402 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:14:00.899727 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 748/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 23 15:14:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:14:01.199871 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7f7e79-d04b-4e14-9a58-be0cc01fe948 None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:14:01.202295 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-6f7f7e79-d04b-4e14-9a58-be0cc01fe948 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:14:01.308477 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7f7e79-d04b-4e14-9a58-be0cc01fe948 None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:14:01.350404 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:14:01.388242 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a8f1f7-e142-4884-bd4a-3fbbfd6f1dc7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd259178570864ffeb936ee10dfec6b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:14:01.388840 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a8f1f7-e142-4884-bd4a-3fbbfd6f1dc7 service neutron] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:14:01.389117 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a8f1f7-e142-4884-bd4a-3fbbfd6f1dc7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:14:01.389910 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a8f1f7-e142-4884-bd4a-3fbbfd6f1dc7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:14:01.391740 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-d1a8f1f7-e142-4884-bd4a-3fbbfd6f1dc7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:14:01.506068 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a8f1f7-e142-4884-bd4a-3fbbfd6f1dc7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:14:01.507515 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a8f1f7-e142-4884-bd4a-3fbbfd6f1dc7 service neutron] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:14:01.528438 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 716/1464] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:14:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:14:03.745514 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49dcdaf-24a6-4556-bc49-8f15fbba4596 None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:14:03.746128 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49dcdaf-24a6-4556-bc49-8f15fbba4596 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:14:03.746246 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49dcdaf-24a6-4556-bc49-8f15fbba4596 None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:14:03.747566 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 749/1465] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 23 15:14:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:14:03.754021 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac7c128-3eec-42a0-935c-e367544b541e None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:14:03.754532 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac7c128-3eec-42a0-935c-e367544b541e None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:14:03.754657 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac7c128-3eec-42a0-935c-e367544b541e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:14:03.824063 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-6ac7c128-3eec-42a0-935c-e367544b541e None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:14:03.825282 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-6ac7c128-3eec-42a0-935c-e367544b541e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:14:03.845433 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:14:03.930439 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 717/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 23 15:14:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 15:14:04.219285 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49dcdaf-24a6-4556-bc49-8f15fbba4596 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:14:04.221000 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-b49dcdaf-24a6-4556-bc49-8f15fbba4596 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:14:04.346543 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49dcdaf-24a6-4556-bc49-8f15fbba4596 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:14:04.421462 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:14:04.428105 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a5c2d7-3387-4eac-872c-ee00001f3c2a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd259178570864ffeb936ee10dfec6b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:14:04.428729 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a5c2d7-3387-4eac-872c-ee00001f3c2a service neutron] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:14:04.429009 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a5c2d7-3387-4eac-872c-ee00001f3c2a service neutron] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:14:04.429279 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a5c2d7-3387-4eac-872c-ee00001f3c2a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:14:04.430911 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-50a5c2d7-3387-4eac-872c-ee00001f3c2a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:14:04.553745 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a5c2d7-3387-4eac-872c-ee00001f3c2a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:14:04.555265 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a5c2d7-3387-4eac-872c-ee00001f3c2a service neutron] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:14:04.575435 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 750/1467] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:14:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 23 15:14:06.766707 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28013b22-c18e-4417-8493-79acc2688e70 None None] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:14:06.767584 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28013b22-c18e-4417-8493-79acc2688e70 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:14:06.767584 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28013b22-c18e-4417-8493-79acc2688e70 None None] PATH_INFO: `/` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:14:06.768888 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 718/1468] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 23 15:14:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:14:06.774194 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172b8862-79ab-434a-8e10-39160382b7a5 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:14:06.774616 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172b8862-79ab-434a-8e10-39160382b7a5 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:14:06.774720 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172b8862-79ab-434a-8e10-39160382b7a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:14:06.840762 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-172b8862-79ab-434a-8e10-39160382b7a5 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:14:06.841974 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-172b8862-79ab-434a-8e10-39160382b7a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:14:06.895615 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:14:06.946595 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 751/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 23 15:14:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:14:07.229840 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28013b22-c18e-4417-8493-79acc2688e70 None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:14:07.231402 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-28013b22-c18e-4417-8493-79acc2688e70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:14:07.341194 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28013b22-c18e-4417-8493-79acc2688e70 None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:14:07.390598 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:14:07.416773 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe9b3b7-40d7-42e5-adef-ab8e937d08c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd259178570864ffeb936ee10dfec6b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:14:07.417496 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe9b3b7-40d7-42e5-adef-ab8e937d08c6 service neutron] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:14:07.417781 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe9b3b7-40d7-42e5-adef-ab8e937d08c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:14:07.418054 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe9b3b7-40d7-42e5-adef-ab8e937d08c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:14:07.419727 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-efe9b3b7-40d7-42e5-adef-ab8e937d08c6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:14:07.529726 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe9b3b7-40d7-42e5-adef-ab8e937d08c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:14:07.532411 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe9b3b7-40d7-42e5-adef-ab8e937d08c6 service neutron] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:14:07.555480 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 719/1470] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:14:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:14:16.945087 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=22 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:14:17.553047 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=66 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:14:28.353232 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cea2fb6-abc2-44ce-97db-5b4e4c9699f2 None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:14:28.354059 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cea2fb6-abc2-44ce-97db-5b4e4c9699f2 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:14:28.354059 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cea2fb6-abc2-44ce-97db-5b4e4c9699f2 None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:14:28.355194 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 752/1471] 10.222.0.55 () {54 vars in 927 bytes} [Fri Sep 23 15:14:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:14:28.361589 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c45066b-1738-48a8-9fb8-1dedad5c6576 None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:14:28.362284 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c45066b-1738-48a8-9fb8-1dedad5c6576 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:14:28.362284 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c45066b-1738-48a8-9fb8-1dedad5c6576 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:14:28.432921 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-7c45066b-1738-48a8-9fb8-1dedad5c6576 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:14:28.434159 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-7c45066b-1738-48a8-9fb8-1dedad5c6576 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:14:28.548973 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 720/1472] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Sep 23 15:14:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:14:28.846285 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd9d12d-5586-4f51-8063-69d5f4d2dda0 None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:14:28.846574 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd9d12d-5586-4f51-8063-69d5f4d2dda0 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:14:28.846854 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd9d12d-5586-4f51-8063-69d5f4d2dda0 None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:14:28.848209 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 753/1473] 10.222.0.55 () {54 vars in 962 bytes} [Fri Sep 23 15:14:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:14:28.853335 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ced4e98-f101-454c-940b-5a34b5afce5a None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:14:28.853619 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ced4e98-f101-454c-940b-5a34b5afce5a None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:14:28.853891 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ced4e98-f101-454c-940b-5a34b5afce5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:14:28.939572 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-5ced4e98-f101-454c-940b-5a34b5afce5a None None] MFA Rules not processed for user `51c39f1d05634589a88c0b6477bd014e`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:14:28.940864 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-5ced4e98-f101-454c-940b-5a34b5afce5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:14:28.965697 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:14:29.027571 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 721/1474] 10.222.0.55 () {58 vars in 1059 bytes} [Fri Sep 23 15:14:28 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:14:29.036050 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd9d12d-5586-4f51-8063-69d5f4d2dda0 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:14:29.037594 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-6cd9d12d-5586-4f51-8063-69d5f4d2dda0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:14:29.149037 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd9d12d-5586-4f51-8063-69d5f4d2dda0 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:14:29.225310 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a9c677-6307-4660-b25e-264916b29ab4 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51c39f1d05634589a88c0b6477bd014e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:14:29.225953 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a9c677-6307-4660-b25e-264916b29ab4 service glance] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:14:29.226234 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a9c677-6307-4660-b25e-264916b29ab4 service glance] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:14:29.226507 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a9c677-6307-4660-b25e-264916b29ab4 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:14:29.228216 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-53a9c677-6307-4660-b25e-264916b29ab4 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:14:29.288116 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:14:29.345587 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a9c677-6307-4660-b25e-264916b29ab4 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:14:29.346929 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a9c677-6307-4660-b25e-264916b29ab4 service glance] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:14:29.365555 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 754/1475] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:14:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:14:31.622598 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee111bb7-7b41-4109-8e4e-45be6def5222 None None] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:14:31.623281 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee111bb7-7b41-4109-8e4e-45be6def5222 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:14:31.623281 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee111bb7-7b41-4109-8e4e-45be6def5222 None None] PATH_INFO: `/` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:14:31.624626 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 722/1476] 10.222.0.55 () {54 vars in 927 bytes} [Fri Sep 23 15:14:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:14:31.630106 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f69f343-8f02-4fa8-b294-178bc3c7f32a None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:14:31.630714 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f69f343-8f02-4fa8-b294-178bc3c7f32a None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:14:31.630714 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f69f343-8f02-4fa8-b294-178bc3c7f32a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:14:31.702593 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-7f69f343-8f02-4fa8-b294-178bc3c7f32a None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:14:31.703819 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-7f69f343-8f02-4fa8-b294-178bc3c7f32a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:14:31.802725 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 755/1477] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Sep 23 15:14:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:14:32.082965 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee111bb7-7b41-4109-8e4e-45be6def5222 None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:14:32.084538 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-ee111bb7-7b41-4109-8e4e-45be6def5222 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:14:32.217281 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee111bb7-7b41-4109-8e4e-45be6def5222 None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:14:32.272281 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:14:32.320219 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bb93d5-313f-4b8e-8ebd-b451fcffbb00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:14:32.320840 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bb93d5-313f-4b8e-8ebd-b451fcffbb00 admin admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:14:32.321123 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bb93d5-313f-4b8e-8ebd-b451fcffbb00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:14:32.321399 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bb93d5-313f-4b8e-8ebd-b451fcffbb00 admin admin] PATH_INFO: `/v3/projects` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:14:32.322051 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81bb93d5-313f-4b8e-8ebd-b451fcffbb00 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:14:32.323955 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: WARNING py.warnings [None req-81bb93d5-313f-4b8e-8ebd-b451fcffbb00 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 23 15:14:32.323955 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: warnings.warn(msg) Sep 23 15:14:32.323955 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: Sep 23 15:14:32.325166 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81bb93d5-313f-4b8e-8ebd-b451fcffbb00 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:14:32.462046 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 723/1478] 10.222.0.55 () {56 vars in 1108 bytes} [Fri Sep 23 15:14:32 2022] GET /identity/v3/projects => generated 2553 bytes in 380 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:14:34.732635 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441b7b6c-5674-46b9-a518-e684b33b1e21 None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:14:34.733632 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441b7b6c-5674-46b9-a518-e684b33b1e21 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:14:34.734430 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441b7b6c-5674-46b9-a518-e684b33b1e21 None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:14:34.736040 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 756/1479] 10.222.0.55 () {54 vars in 927 bytes} [Fri Sep 23 15:14:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:14:34.744426 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c5368d-d340-4934-ad12-ec67fb05d29c None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:14:34.745021 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c5368d-d340-4934-ad12-ec67fb05d29c None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:14:34.745021 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c5368d-d340-4934-ad12-ec67fb05d29c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:14:34.820632 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-90c5368d-d340-4934-ad12-ec67fb05d29c None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:14:34.822002 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-90c5368d-d340-4934-ad12-ec67fb05d29c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:14:34.961054 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 724/1480] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Sep 23 15:14:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:14:35.278066 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441b7b6c-5674-46b9-a518-e684b33b1e21 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:14:35.280161 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-441b7b6c-5674-46b9-a518-e684b33b1e21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:14:35.297398 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:14:35.455638 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441b7b6c-5674-46b9-a518-e684b33b1e21 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:14:35.552295 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:14:35.589567 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e85ec67-29fd-45ca-83c8-348c7bde29a1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3c6c7a1325a47be924268e1aa1a7a18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:14:35.590178 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e85ec67-29fd-45ca-83c8-348c7bde29a1 service nova] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:14:35.590463 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e85ec67-29fd-45ca-83c8-348c7bde29a1 service nova] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:14:35.590741 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e85ec67-29fd-45ca-83c8-348c7bde29a1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:14:35.592615 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-5e85ec67-29fd-45ca-83c8-348c7bde29a1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:14:35.707352 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e85ec67-29fd-45ca-83c8-348c7bde29a1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:14:35.708939 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e85ec67-29fd-45ca-83c8-348c7bde29a1 service nova] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:14:35.729041 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 757/1481] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:14:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 451 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:14:38.187297 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb07479-456d-4d25-9e49-e33e98d4e313 None None] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:14:38.187870 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb07479-456d-4d25-9e49-e33e98d4e313 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:14:38.188035 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb07479-456d-4d25-9e49-e33e98d4e313 None None] PATH_INFO: `/` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:14:38.189545 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 725/1482] 10.222.0.55 () {54 vars in 927 bytes} [Fri Sep 23 15:14:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 23 15:14:38.195681 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf53cfb-913c-4c20-93ca-de55a1d125d1 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:14:38.196342 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf53cfb-913c-4c20-93ca-de55a1d125d1 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:14:38.196342 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf53cfb-913c-4c20-93ca-de55a1d125d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:14:38.275739 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-ebf53cfb-913c-4c20-93ca-de55a1d125d1 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:14:38.277012 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-ebf53cfb-913c-4c20-93ca-de55a1d125d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:14:38.331568 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:14:38.388561 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 758/1483] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Sep 23 15:14:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 15:14:38.684437 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb07479-456d-4d25-9e49-e33e98d4e313 None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:14:38.686746 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-8eb07479-456d-4d25-9e49-e33e98d4e313 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:14:38.714510 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:14:38.833597 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb07479-456d-4d25-9e49-e33e98d4e313 None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:14:38.932941 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0f02e2-2e8e-4283-b8d6-b6ddab2be833 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd259178570864ffeb936ee10dfec6b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:14:38.933475 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0f02e2-2e8e-4283-b8d6-b6ddab2be833 service neutron] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:14:38.933750 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0f02e2-2e8e-4283-b8d6-b6ddab2be833 service neutron] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:14:38.934017 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0f02e2-2e8e-4283-b8d6-b6ddab2be833 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:14:38.935704 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-4a0f02e2-2e8e-4283-b8d6-b6ddab2be833 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:14:39.037884 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:14:39.095697 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0f02e2-2e8e-4283-b8d6-b6ddab2be833 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:14:39.098504 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0f02e2-2e8e-4283-b8d6-b6ddab2be833 service neutron] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:14:39.134172 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 726/1484] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:14:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 450 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 23 15:14:41.548785 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff2c1f0-91aa-4637-b900-8e566221f0b8 None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:14:41.549404 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff2c1f0-91aa-4637-b900-8e566221f0b8 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:14:41.549404 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff2c1f0-91aa-4637-b900-8e566221f0b8 None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:14:41.550938 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 759/1485] 10.222.0.55 () {54 vars in 927 bytes} [Fri Sep 23 15:14:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 23 15:14:41.557198 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98372b70-8505-4bf9-9ec9-f6fd7e345d45 None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:14:41.557774 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98372b70-8505-4bf9-9ec9-f6fd7e345d45 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:14:41.557774 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98372b70-8505-4bf9-9ec9-f6fd7e345d45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:14:41.637922 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-98372b70-8505-4bf9-9ec9-f6fd7e345d45 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:14:41.640599 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-98372b70-8505-4bf9-9ec9-f6fd7e345d45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:14:41.764740 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 727/1486] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Sep 23 15:14:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 15:14:42.086205 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff2c1f0-91aa-4637-b900-8e566221f0b8 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:14:42.087850 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-dff2c1f0-91aa-4637-b900-8e566221f0b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:14:42.210563 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff2c1f0-91aa-4637-b900-8e566221f0b8 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:14:42.293122 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:14:42.294851 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0eb4b31-d000-4486-b932-1b81573d372d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd259178570864ffeb936ee10dfec6b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:14:42.295800 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0eb4b31-d000-4486-b932-1b81573d372d service neutron] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:14:42.296115 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0eb4b31-d000-4486-b932-1b81573d372d service neutron] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:14:42.296393 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0eb4b31-d000-4486-b932-1b81573d372d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:14:42.298064 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-b0eb4b31-d000-4486-b932-1b81573d372d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:14:42.419398 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0eb4b31-d000-4486-b932-1b81573d372d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:14:42.421030 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0eb4b31-d000-4486-b932-1b81573d372d service neutron] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:14:42.441474 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 760/1487] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:14:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:14:44.873458 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83570bfe-4f4d-40d7-a2aa-22020ebb894a None None] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:14:44.874338 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83570bfe-4f4d-40d7-a2aa-22020ebb894a None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:14:44.874338 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83570bfe-4f4d-40d7-a2aa-22020ebb894a None None] PATH_INFO: `/` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:14:44.876362 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 728/1488] 10.222.0.55 () {54 vars in 927 bytes} [Fri Sep 23 15:14:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 23 15:14:44.882444 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b5ef4d-69e5-4b6e-a0d2-42e231db792c None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:14:44.883229 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b5ef4d-69e5-4b6e-a0d2-42e231db792c None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:14:44.883419 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b5ef4d-69e5-4b6e-a0d2-42e231db792c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:14:44.960336 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-c3b5ef4d-69e5-4b6e-a0d2-42e231db792c None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:14:44.961588 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-c3b5ef4d-69e5-4b6e-a0d2-42e231db792c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:14:45.020897 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:14:45.070209 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 761/1489] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Sep 23 15:14:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:14:45.377334 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83570bfe-4f4d-40d7-a2aa-22020ebb894a None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:14:45.379130 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-83570bfe-4f4d-40d7-a2aa-22020ebb894a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:14:45.423021 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:14:45.505553 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83570bfe-4f4d-40d7-a2aa-22020ebb894a None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:14:45.600279 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac51806-072d-4302-b381-f443a1e6343e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd259178570864ffeb936ee10dfec6b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:14:45.600946 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac51806-072d-4302-b381-f443a1e6343e service neutron] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:14:45.601226 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac51806-072d-4302-b381-f443a1e6343e service neutron] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:14:45.601487 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac51806-072d-4302-b381-f443a1e6343e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:14:45.604391 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-3ac51806-072d-4302-b381-f443a1e6343e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:14:45.713447 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:14:45.751248 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac51806-072d-4302-b381-f443a1e6343e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:14:45.752740 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac51806-072d-4302-b381-f443a1e6343e service neutron] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:14:45.773634 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 729/1490] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:14:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 23 15:14:48.632048 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d5113a-a56d-4f9b-9b52-e79027f99937 None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:14:48.632881 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d5113a-a56d-4f9b-9b52-e79027f99937 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:14:48.632881 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d5113a-a56d-4f9b-9b52-e79027f99937 None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:14:48.634278 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 762/1491] 10.222.0.55 () {54 vars in 927 bytes} [Fri Sep 23 15:14:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 23 15:14:48.644306 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f52bb0e-5258-4ab9-b0b1-5dc49e8b0457 None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:14:48.644907 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f52bb0e-5258-4ab9-b0b1-5dc49e8b0457 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:14:48.645117 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f52bb0e-5258-4ab9-b0b1-5dc49e8b0457 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:14:48.738896 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-1f52bb0e-5258-4ab9-b0b1-5dc49e8b0457 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:14:48.740301 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-1f52bb0e-5258-4ab9-b0b1-5dc49e8b0457 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:14:48.859454 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 730/1492] 10.222.0.55 () {58 vars in 1024 bytes} [Fri Sep 23 15:14:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:14:49.155621 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d5113a-a56d-4f9b-9b52-e79027f99937 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:14:49.157232 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-30d5113a-a56d-4f9b-9b52-e79027f99937 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:14:49.275683 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d5113a-a56d-4f9b-9b52-e79027f99937 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:14:49.366180 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f63e5f3-abb6-4d25-a6fa-c5bc0a83bcb7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd259178570864ffeb936ee10dfec6b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:14:49.366690 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f63e5f3-abb6-4d25-a6fa-c5bc0a83bcb7 service neutron] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:14:49.366972 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f63e5f3-abb6-4d25-a6fa-c5bc0a83bcb7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:14:49.367246 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f63e5f3-abb6-4d25-a6fa-c5bc0a83bcb7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:14:49.369329 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-8f63e5f3-abb6-4d25-a6fa-c5bc0a83bcb7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:14:49.381613 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:14:49.513132 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f63e5f3-abb6-4d25-a6fa-c5bc0a83bcb7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:14:49.514758 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f63e5f3-abb6-4d25-a6fa-c5bc0a83bcb7 service neutron] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:14:49.536086 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 763/1493] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:14:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 23 15:14:58.857166 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=82 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:14:59.533647 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=50 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:16:20.174991 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5870b1be-9513-4160-a3fe-8598638f008c None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:16:20.175601 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5870b1be-9513-4160-a3fe-8598638f008c None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:16:20.175709 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5870b1be-9513-4160-a3fe-8598638f008c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:16:20.265864 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-5870b1be-9513-4160-a3fe-8598638f008c None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:16:20.267351 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-5870b1be-9513-4160-a3fe-8598638f008c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:16:20.383554 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 731/1494] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:16:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 15:16:20.396787 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1ac78d-8c4c-4524-a772-c6ddd9f044cc None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:16:20.397415 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1ac78d-8c4c-4524-a772-c6ddd9f044cc None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:16:20.397415 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1ac78d-8c4c-4524-a772-c6ddd9f044cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:16:20.505879 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-2a1ac78d-8c4c-4524-a772-c6ddd9f044cc None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:16:20.507151 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-2a1ac78d-8c4c-4524-a772-c6ddd9f044cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:16:20.627043 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 764/1495] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:16:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:16:20.632032 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5870b1be-9513-4160-a3fe-8598638f008c None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:16:20.633339 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-5870b1be-9513-4160-a3fe-8598638f008c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:16:20.647244 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5870b1be-9513-4160-a3fe-8598638f008c None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:16:20.715684 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:16:20.758637 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a2fd7c-8ab9-4023-b404-785f87385f34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:16:20.759679 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a2fd7c-8ab9-4023-b404-785f87385f34 admin admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:16:20.760174 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a2fd7c-8ab9-4023-b404-785f87385f34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:16:20.760470 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a2fd7c-8ab9-4023-b404-785f87385f34 admin admin] PATH_INFO: `/v3/domains` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:16:20.761662 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a2fd7c-8ab9-4023-b404-785f87385f34 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:16:20.762209 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a2fd7c-8ab9-4023-b404-785f87385f34 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:16:20.764684 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: WARNING py.warnings [None req-d4a2fd7c-8ab9-4023-b404-785f87385f34 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 23 15:16:20.764684 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: warnings.warn(msg) Sep 23 15:16:20.764684 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: Sep 23 15:16:20.766376 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a2fd7c-8ab9-4023-b404-785f87385f34 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:16:20.787748 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 732/1496] 10.222.0.55 () {58 vars in 1153 bytes} [Fri Sep 23 15:16:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 15:16:20.797295 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1ac78d-8c4c-4524-a772-c6ddd9f044cc None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:16:20.807411 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1ac78d-8c4c-4524-a772-c6ddd9f044cc None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:16:20.813332 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9733c223-ffcd-47c0-b71a-e3438a6ac91f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:16:20.813978 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9733c223-ffcd-47c0-b71a-e3438a6ac91f admin admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:16:20.814263 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9733c223-ffcd-47c0-b71a-e3438a6ac91f admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:16:20.814539 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9733c223-ffcd-47c0-b71a-e3438a6ac91f admin admin] PATH_INFO: `/v3/projects` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:16:20.815201 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9733c223-ffcd-47c0-b71a-e3438a6ac91f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:16:20.817103 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: WARNING py.warnings [None req-9733c223-ffcd-47c0-b71a-e3438a6ac91f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 23 15:16:20.817103 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: warnings.warn(msg) Sep 23 15:16:20.817103 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: Sep 23 15:16:20.818337 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9733c223-ffcd-47c0-b71a-e3438a6ac91f admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:16:20.945410 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 765/1497] 10.222.0.55 () {60 vars in 1153 bytes} [Fri Sep 23 15:16:20 2022] POST /identity/v3/projects => generated 377 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 23 15:16:20.952674 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a2fd7c-8ab9-4023-b404-785f87385f34 admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:16:20.968852 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a2fd7c-8ab9-4023-b404-785f87385f34 admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:16:21.069142 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b0850e-dda4-4d94-8aa9-3ded4bf28e67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:16:21.069771 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b0850e-dda4-4d94-8aa9-3ded4bf28e67 admin admin] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:16:21.070047 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b0850e-dda4-4d94-8aa9-3ded4bf28e67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:16:21.070316 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b0850e-dda4-4d94-8aa9-3ded4bf28e67 admin admin] PATH_INFO: `/v3/users` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:16:21.071008 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b0850e-dda4-4d94-8aa9-3ded4bf28e67 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:16:21.072089 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: WARNING py.warnings [None req-81b0850e-dda4-4d94-8aa9-3ded4bf28e67 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 23 15:16:21.072089 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: warnings.warn(msg) Sep 23 15:16:21.072089 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: Sep 23 15:16:21.073345 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b0850e-dda4-4d94-8aa9-3ded4bf28e67 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:16:21.174386 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 733/1498] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Sep 23 15:16:20 2022] POST /identity/v3/users => generated 289 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:16:21.178772 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9733c223-ffcd-47c0-b71a-e3438a6ac91f admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:16:21.191393 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9733c223-ffcd-47c0-b71a-e3438a6ac91f admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:16:21.196603 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14774f88-63fe-4822-8644-697107f0ed28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:16:21.197186 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14774f88-63fe-4822-8644-697107f0ed28 admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:16:21.197465 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14774f88-63fe-4822-8644-697107f0ed28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:16:21.197741 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14774f88-63fe-4822-8644-697107f0ed28 admin admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:16:21.198402 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14774f88-63fe-4822-8644-697107f0ed28 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:16:21.199429 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: WARNING py.warnings [None req-14774f88-63fe-4822-8644-697107f0ed28 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 23 15:16:21.199429 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: warnings.warn(msg) Sep 23 15:16:21.199429 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: Sep 23 15:16:21.200409 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14774f88-63fe-4822-8644-697107f0ed28 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:16:21.217694 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 766/1499] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Sep 23 15:16:21 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 23 15:16:21.222428 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b0850e-dda4-4d94-8aa9-3ded4bf28e67 admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:16:21.237936 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b0850e-dda4-4d94-8aa9-3ded4bf28e67 admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:16:21.239667 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6416c93-96b1-45b9-ab95-5bcd69cef00a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:16:21.240398 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6416c93-96b1-45b9-ab95-5bcd69cef00a admin admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:16:21.244064 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6416c93-96b1-45b9-ab95-5bcd69cef00a admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:16:21.244064 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6416c93-96b1-45b9-ab95-5bcd69cef00a admin admin] PATH_INFO: `/v3/roles` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:16:21.244064 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6416c93-96b1-45b9-ab95-5bcd69cef00a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:16:21.244064 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: WARNING py.warnings [None req-c6416c93-96b1-45b9-ab95-5bcd69cef00a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 23 15:16:21.244064 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: warnings.warn(msg) Sep 23 15:16:21.244064 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: Sep 23 15:16:21.244064 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6416c93-96b1-45b9-ab95-5bcd69cef00a admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:16:21.260051 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 734/1500] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Sep 23 15:16:21 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:16:21.265992 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14774f88-63fe-4822-8644-697107f0ed28 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:16:21.283457 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14774f88-63fe-4822-8644-697107f0ed28 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:16:21.291082 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25464fa8-b60f-4280-b7f0-8bc0fdf361c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:16:21.292154 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25464fa8-b60f-4280-b7f0-8bc0fdf361c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:16:21.292596 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25464fa8-b60f-4280-b7f0-8bc0fdf361c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:16:21.293091 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25464fa8-b60f-4280-b7f0-8bc0fdf361c1 admin admin] PATH_INFO: `/v3/projects/caea237487d64e74bddfb816b7678bb5/users/ba70a99119714f7a8f74fcd13b58a9a4/roles/af257be2dee54d6b942f58650196582c` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:16:21.312834 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25464fa8-b60f-4280-b7f0-8bc0fdf361c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=caea237487d64e74bddfb816b7678bb5, user_id=ba70a99119714f7a8f74fcd13b58a9a4, role_id=af257be2dee54d6b942f58650196582c)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:16:21.313880 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: WARNING py.warnings [None req-25464fa8-b60f-4280-b7f0-8bc0fdf361c1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 23 15:16:21.313880 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: warnings.warn(msg) Sep 23 15:16:21.313880 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: Sep 23 15:16:21.315810 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25464fa8-b60f-4280-b7f0-8bc0fdf361c1 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:16:21.385909 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 767/1501] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Sep 23 15:16:21 2022] PUT /identity/v3/projects/caea237487d64e74bddfb816b7678bb5/users/ba70a99119714f7a8f74fcd13b58a9a4/roles/af257be2dee54d6b942f58650196582c => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 15:16:21.408016 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25214ea-35db-4d71-b410-46095fb5ae01 None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:16:21.408506 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25214ea-35db-4d71-b410-46095fb5ae01 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:16:21.408995 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25214ea-35db-4d71-b410-46095fb5ae01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:16:21.450091 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-c25214ea-35db-4d71-b410-46095fb5ae01 None None] MFA Rules not processed for user `ba70a99119714f7a8f74fcd13b58a9a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:16:21.451372 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-c25214ea-35db-4d71-b410-46095fb5ae01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:16:21.534519 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 735/1502] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:16:21 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:16:21.550311 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636fe7d4-c3c5-4677-bae5-414ffd93f2b9 None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:16:21.550311 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636fe7d4-c3c5-4677-bae5-414ffd93f2b9 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:16:21.550311 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636fe7d4-c3c5-4677-bae5-414ffd93f2b9 None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:16:21.552480 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 768/1503] 10.222.0.55 () {54 vars in 963 bytes} [Fri Sep 23 15:16:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:16:21.558259 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd406734-498f-4e5d-b3fc-52214e7abec9 None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:16:21.558546 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd406734-498f-4e5d-b3fc-52214e7abec9 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:16:21.558821 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd406734-498f-4e5d-b3fc-52214e7abec9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:16:21.606157 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:16:21.632953 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-dd406734-498f-4e5d-b3fc-52214e7abec9 None None] MFA Rules not processed for user `feca6278be904ef8b0dc580a67e477f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:16:21.634203 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-dd406734-498f-4e5d-b3fc-52214e7abec9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:16:21.726740 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 736/1504] 10.222.0.55 () {58 vars in 1060 bytes} [Fri Sep 23 15:16:21 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:16:21.738558 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636fe7d4-c3c5-4677-bae5-414ffd93f2b9 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:16:21.744212 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:16:21.751464 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636fe7d4-c3c5-4677-bae5-414ffd93f2b9 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:16:21.755203 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9e5677-6734-4a31-9395-c68a0536726a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feca6278be904ef8b0dc580a67e477f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:16:21.755840 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9e5677-6734-4a31-9395-c68a0536726a service cinder] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:16:21.756181 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9e5677-6734-4a31-9395-c68a0536726a service cinder] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:16:21.756558 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9e5677-6734-4a31-9395-c68a0536726a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:16:21.766295 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9e5677-6734-4a31-9395-c68a0536726a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:16:21.768036 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9e5677-6734-4a31-9395-c68a0536726a service cinder] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:16:21.778347 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 769/1505] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:16:21 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 23 15:16:21.839880 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7d253e-b0e7-417d-a8b5-944d906d1c1d None None] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:16:21.840212 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7d253e-b0e7-417d-a8b5-944d906d1c1d None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:16:21.840516 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7d253e-b0e7-417d-a8b5-944d906d1c1d None None] PATH_INFO: `/` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:16:21.841819 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 737/1506] 10.222.0.55 () {52 vars in 830 bytes} [Fri Sep 23 15:16:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:16:21.846240 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9e5677-6734-4a31-9395-c68a0536726a service cinder] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:16:21.941724 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9e5677-6734-4a31-9395-c68a0536726a service cinder] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:16:21.946970 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6a5392-5a5b-4ca9-a15a-e858ca8997ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:16:21.948074 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6a5392-5a5b-4ca9-a15a-e858ca8997ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:16:21.948457 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6a5392-5a5b-4ca9-a15a-e858ca8997ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:16:21.948799 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6a5392-5a5b-4ca9-a15a-e858ca8997ce admin admin] PATH_INFO: `/v3/users/ba70a99119714f7a8f74fcd13b58a9a4` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:16:21.950173 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6a5392-5a5b-4ca9-a15a-e858ca8997ce admin admin] RBAC: Authorizing `identity:delete_user(user_id=ba70a99119714f7a8f74fcd13b58a9a4)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:16:21.953970 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: WARNING py.warnings [None req-4f6a5392-5a5b-4ca9-a15a-e858ca8997ce admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 23 15:16:21.953970 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: warnings.warn(msg) Sep 23 15:16:21.953970 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: Sep 23 15:16:21.955568 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6a5392-5a5b-4ca9-a15a-e858ca8997ce admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:16:22.210426 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-4f6a5392-5a5b-4ca9-a15a-e858ca8997ce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ba70a99119714f7a8f74fcd13b58a9a4'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:16:22.244930 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-4f6a5392-5a5b-4ca9-a15a-e858ca8997ce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ba70a99119714f7a8f74fcd13b58a9a4'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:16:22.262934 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-4f6a5392-5a5b-4ca9-a15a-e858ca8997ce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ba70a99119714f7a8f74fcd13b58a9a4'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:16:22.350928 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-4f6a5392-5a5b-4ca9-a15a-e858ca8997ce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ba70a99119714f7a8f74fcd13b58a9a4'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:16:22.351591 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-4f6a5392-5a5b-4ca9-a15a-e858ca8997ce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ba70a99119714f7a8f74fcd13b58a9a4'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:16:22.361702 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 770/1507] 10.222.0.55 () {58 vars in 1224 bytes} [Fri Sep 23 15:16:21 2022] DELETE /identity/v3/users/ba70a99119714f7a8f74fcd13b58a9a4 => generated 0 bytes in 516 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:16:22.367444 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c05629a-a785-44d1-b928-2a3c3049a2a0 None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:16:22.368131 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c05629a-a785-44d1-b928-2a3c3049a2a0 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:16:22.368131 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c05629a-a785-44d1-b928-2a3c3049a2a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:16:22.388540 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-5c05629a-a785-44d1-b928-2a3c3049a2a0 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:16:22.389989 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-5c05629a-a785-44d1-b928-2a3c3049a2a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:16:22.453880 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 738/1508] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:16:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 15:16:22.463897 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6a5392-5a5b-4ca9-a15a-e858ca8997ce admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:16:22.465439 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-4f6a5392-5a5b-4ca9-a15a-e858ca8997ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:16:22.563778 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:16:22.568010 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6a5392-5a5b-4ca9-a15a-e858ca8997ce admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:16:22.571916 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae34328a-21a2-408b-9005-d518b49f3c02 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd259178570864ffeb936ee10dfec6b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:16:22.572582 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae34328a-21a2-408b-9005-d518b49f3c02 service neutron] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:16:22.572868 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae34328a-21a2-408b-9005-d518b49f3c02 service neutron] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:16:22.573149 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae34328a-21a2-408b-9005-d518b49f3c02 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:16:22.583005 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae34328a-21a2-408b-9005-d518b49f3c02 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:16:22.584520 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae34328a-21a2-408b-9005-d518b49f3c02 service neutron] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:16:22.595460 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 771/1509] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:16:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 23 15:16:23.205469 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c05629a-a785-44d1-b928-2a3c3049a2a0 None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:16:23.207162 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-5c05629a-a785-44d1-b928-2a3c3049a2a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:16:23.222900 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c05629a-a785-44d1-b928-2a3c3049a2a0 None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:16:23.230010 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc13d604-b46b-4914-9f18-4c44705900e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:16:23.230010 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc13d604-b46b-4914-9f18-4c44705900e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:16:23.230010 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc13d604-b46b-4914-9f18-4c44705900e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:16:23.230010 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc13d604-b46b-4914-9f18-4c44705900e7 admin admin] PATH_INFO: `/v3/projects/caea237487d64e74bddfb816b7678bb5` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:16:23.230582 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc13d604-b46b-4914-9f18-4c44705900e7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=caea237487d64e74bddfb816b7678bb5)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:16:23.232005 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: WARNING py.warnings [None req-cc13d604-b46b-4914-9f18-4c44705900e7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 23 15:16:23.232005 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: warnings.warn(msg) Sep 23 15:16:23.232005 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: Sep 23 15:16:23.233263 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc13d604-b46b-4914-9f18-4c44705900e7 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:16:23.463784 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-cc13d604-b46b-4914-9f18-4c44705900e7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'caea237487d64e74bddfb816b7678bb5', 'request_id': 'req-cc13d604-b46b-4914-9f18-4c44705900e7'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:16:23.474315 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-cc13d604-b46b-4914-9f18-4c44705900e7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'caea237487d64e74bddfb816b7678bb5', 'request_id': 'req-cc13d604-b46b-4914-9f18-4c44705900e7'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:16:23.583546 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-cc13d604-b46b-4914-9f18-4c44705900e7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'caea237487d64e74bddfb816b7678bb5', 'request_id': 'req-cc13d604-b46b-4914-9f18-4c44705900e7'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:16:23.604208 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-cc13d604-b46b-4914-9f18-4c44705900e7 admin admin] The token cache is being invalidate because project caea237487d64e74bddfb816b7678bb5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81993) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 15:16:23.604500 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-cc13d604-b46b-4914-9f18-4c44705900e7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:16:23.605164 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-cc13d604-b46b-4914-9f18-4c44705900e7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:16:23.606764 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 739/1510] 10.222.0.55 () {58 vars in 1233 bytes} [Fri Sep 23 15:16:23 2022] DELETE /identity/v3/projects/caea237487d64e74bddfb816b7678bb5 => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:16:32.739003 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=4 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:16:33.600878 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:17:58.696443 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e3c714-535c-4d9b-9346-b4e61ea91f59 None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:17:58.697054 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e3c714-535c-4d9b-9346-b4e61ea91f59 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:17:58.697054 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e3c714-535c-4d9b-9346-b4e61ea91f59 None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:17:58.698430 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 772/1511] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 23 15:17:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:17:58.704167 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb12113-05f0-4053-8931-3118cd5bf89b None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:17:58.704699 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb12113-05f0-4053-8931-3118cd5bf89b None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:17:58.704699 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb12113-05f0-4053-8931-3118cd5bf89b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:17:58.724924 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-6fb12113-05f0-4053-8931-3118cd5bf89b None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:17:58.726122 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-6fb12113-05f0-4053-8931-3118cd5bf89b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:17:58.788791 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 740/1512] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 23 15:17:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:17:58.947351 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e3c714-535c-4d9b-9346-b4e61ea91f59 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:17:58.949097 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-08e3c714-535c-4d9b-9346-b4e61ea91f59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:17:59.070479 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e3c714-535c-4d9b-9346-b4e61ea91f59 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:17:59.077052 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9833b5-7999-4755-8f8a-0cc333d1c747 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3c6c7a1325a47be924268e1aa1a7a18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:17:59.078981 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9833b5-7999-4755-8f8a-0cc333d1c747 service nova] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:17:59.078981 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9833b5-7999-4755-8f8a-0cc333d1c747 service nova] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:17:59.078981 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9833b5-7999-4755-8f8a-0cc333d1c747 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:17:59.087676 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9833b5-7999-4755-8f8a-0cc333d1c747 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:17:59.089348 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9833b5-7999-4755-8f8a-0cc333d1c747 service nova] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:17:59.101031 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 773/1513] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:17:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:18:00.280395 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b6a1b0-9189-4cc8-9a83-f01b5a20234e None None] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:00.280972 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b6a1b0-9189-4cc8-9a83-f01b5a20234e None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:00.280972 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b6a1b0-9189-4cc8-9a83-f01b5a20234e None None] PATH_INFO: `/` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:00.282389 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 741/1514] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 23 15:18:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 23 15:18:00.289063 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82615a31-409c-436b-aff2-35b5b4654680 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:00.289581 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82615a31-409c-436b-aff2-35b5b4654680 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:00.289694 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82615a31-409c-436b-aff2-35b5b4654680 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:00.312697 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-82615a31-409c-436b-aff2-35b5b4654680 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:18:00.313913 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-82615a31-409c-436b-aff2-35b5b4654680 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:18:00.322256 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 774/1515] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 23 15:18:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:18:00.474538 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b6a1b0-9189-4cc8-9a83-f01b5a20234e None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:18:00.489796 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b6a1b0-9189-4cc8-9a83-f01b5a20234e None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:18:00.495247 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b3d08a-f36a-4570-a5e7-040d4165641b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3c6c7a1325a47be924268e1aa1a7a18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:18:00.495847 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b3d08a-f36a-4570-a5e7-040d4165641b service nova] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:00.496157 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b3d08a-f36a-4570-a5e7-040d4165641b service nova] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:00.496434 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b3d08a-f36a-4570-a5e7-040d4165641b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:00.505326 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b3d08a-f36a-4570-a5e7-040d4165641b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:18:00.507211 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b3d08a-f36a-4570-a5e7-040d4165641b service nova] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:18:00.508857 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ef654a-66b5-4d61-8b28-e6db9b02625e None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:00.509144 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ef654a-66b5-4d61-8b28-e6db9b02625e None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:00.509427 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ef654a-66b5-4d61-8b28-e6db9b02625e None None] PATH_INFO: `/v3` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:00.510709 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 775/1516] 10.222.0.77 () {54 vars in 925 bytes} [Fri Sep 23 15:18:00 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 15:18:00.517551 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b533c0d-f061-4ee9-af86-d102d74b16b5 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:00.517832 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b533c0d-f061-4ee9-af86-d102d74b16b5 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:00.518369 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b533c0d-f061-4ee9-af86-d102d74b16b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:00.518528 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 742/1517] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:18:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:18:00.573631 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-4b533c0d-f061-4ee9-af86-d102d74b16b5 None None] MFA Rules not processed for user `09f1a55d180642c781a6e0b017d9f0aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:18:00.574941 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-4b533c0d-f061-4ee9-af86-d102d74b16b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:18:00.664348 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 776/1518] 10.222.0.77 () {58 vars in 1015 bytes} [Fri Sep 23 15:18:00 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 15:18:04.561273 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb80e5b-4f7c-4e95-9ebf-c502a334d513 None None] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:04.561879 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb80e5b-4f7c-4e95-9ebf-c502a334d513 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:04.562053 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb80e5b-4f7c-4e95-9ebf-c502a334d513 None None] PATH_INFO: `/` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:04.564124 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 743/1519] 10.222.0.55 () {54 vars in 961 bytes} [Fri Sep 23 15:18:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:18:04.573348 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e97aa7d-7bd0-498b-9dbd-31911df86a90 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:04.573348 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e97aa7d-7bd0-498b-9dbd-31911df86a90 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:04.573348 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e97aa7d-7bd0-498b-9dbd-31911df86a90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:04.590584 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-9e97aa7d-7bd0-498b-9dbd-31911df86a90 None None] MFA Rules not processed for user `09f1a55d180642c781a6e0b017d9f0aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:18:04.592710 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-9e97aa7d-7bd0-498b-9dbd-31911df86a90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:18:04.601646 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 777/1520] 10.222.0.55 () {58 vars in 1058 bytes} [Fri Sep 23 15:18:04 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:18:04.610246 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb80e5b-4f7c-4e95-9ebf-c502a334d513 None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:18:04.624497 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb80e5b-4f7c-4e95-9ebf-c502a334d513 None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:18:04.629766 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73ea817-1671-4b7a-9b98-ffe09481e826 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09f1a55d180642c781a6e0b017d9f0aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:18:04.630351 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73ea817-1671-4b7a-9b98-ffe09481e826 service placement] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:04.630635 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73ea817-1671-4b7a-9b98-ffe09481e826 service placement] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:04.630909 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73ea817-1671-4b7a-9b98-ffe09481e826 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:04.640958 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73ea817-1671-4b7a-9b98-ffe09481e826 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:18:04.642587 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73ea817-1671-4b7a-9b98-ffe09481e826 service placement] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:18:04.653336 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 744/1521] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:18:04 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:18:12.055101 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969e9d0c-1660-4fee-b0c3-223d4d007ef4 None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:12.055667 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969e9d0c-1660-4fee-b0c3-223d4d007ef4 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:12.055667 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969e9d0c-1660-4fee-b0c3-223d4d007ef4 None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:12.056963 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 778/1522] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 23 15:18:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:18:12.064250 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1752c2-1938-4dfb-9869-7e6b29f35b2e None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:12.064793 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1752c2-1938-4dfb-9869-7e6b29f35b2e None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:12.064793 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1752c2-1938-4dfb-9869-7e6b29f35b2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:12.085376 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-4e1752c2-1938-4dfb-9869-7e6b29f35b2e None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:18:12.086585 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-4e1752c2-1938-4dfb-9869-7e6b29f35b2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:18:12.095151 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 745/1523] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 23 15:18:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:18:12.249370 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969e9d0c-1660-4fee-b0c3-223d4d007ef4 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:18:12.263968 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969e9d0c-1660-4fee-b0c3-223d4d007ef4 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:18:12.269897 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b1bc15-165f-4d7e-a711-4f7c4199585c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3c6c7a1325a47be924268e1aa1a7a18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:18:12.270506 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b1bc15-165f-4d7e-a711-4f7c4199585c service nova] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:12.270794 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b1bc15-165f-4d7e-a711-4f7c4199585c service nova] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:12.271071 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b1bc15-165f-4d7e-a711-4f7c4199585c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:12.277485 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:18:12.280658 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b1bc15-165f-4d7e-a711-4f7c4199585c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:18:12.282252 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b1bc15-165f-4d7e-a711-4f7c4199585c service nova] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:18:12.292054 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 779/1524] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:18:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:18:13.409750 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9ef299-c127-41f0-8806-d7b19f39cc30 None None] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:13.410321 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9ef299-c127-41f0-8806-d7b19f39cc30 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:13.410321 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9ef299-c127-41f0-8806-d7b19f39cc30 None None] PATH_INFO: `/` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:13.411669 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 746/1525] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 23 15:18:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:18:13.418481 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71155931-d4a3-450f-8f29-215c2517531d None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:13.418983 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71155931-d4a3-450f-8f29-215c2517531d None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:13.419096 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71155931-d4a3-450f-8f29-215c2517531d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:13.438853 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-71155931-d4a3-450f-8f29-215c2517531d None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:18:13.440089 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-71155931-d4a3-450f-8f29-215c2517531d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:18:13.448049 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 780/1526] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 23 15:18:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:18:13.601985 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9ef299-c127-41f0-8806-d7b19f39cc30 None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:18:13.617825 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9ef299-c127-41f0-8806-d7b19f39cc30 None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:18:13.623917 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9453f67a-4606-4e74-9127-89a40be23a16 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3c6c7a1325a47be924268e1aa1a7a18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:18:13.624555 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9453f67a-4606-4e74-9127-89a40be23a16 service nova] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:13.624851 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9453f67a-4606-4e74-9127-89a40be23a16 service nova] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:13.625127 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9453f67a-4606-4e74-9127-89a40be23a16 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:13.633461 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9453f67a-4606-4e74-9127-89a40be23a16 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:18:13.635097 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9453f67a-4606-4e74-9127-89a40be23a16 service nova] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:18:13.646258 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 747/1527] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:18:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:18:18.110732 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abc2c04-62ce-4405-8db3-ea47dddb2836 None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:18.111338 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abc2c04-62ce-4405-8db3-ea47dddb2836 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:18.111338 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abc2c04-62ce-4405-8db3-ea47dddb2836 None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:18.112662 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 781/1528] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 23 15:18:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:18:18.118830 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52f7a7f-e1af-4eaf-b61b-1ff27707c9c0 None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:18.119359 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52f7a7f-e1af-4eaf-b61b-1ff27707c9c0 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:18.119465 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52f7a7f-e1af-4eaf-b61b-1ff27707c9c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:18.140929 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-a52f7a7f-e1af-4eaf-b61b-1ff27707c9c0 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:18:18.142170 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-a52f7a7f-e1af-4eaf-b61b-1ff27707c9c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:18:18.151528 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 748/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 23 15:18:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:18:18.440908 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abc2c04-62ce-4405-8db3-ea47dddb2836 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:18:18.442672 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-8abc2c04-62ce-4405-8db3-ea47dddb2836 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:18:18.503471 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abc2c04-62ce-4405-8db3-ea47dddb2836 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:18:18.507079 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d14b01-2a49-4798-b78d-42ccdc6f36c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd259178570864ffeb936ee10dfec6b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:18:18.507701 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d14b01-2a49-4798-b78d-42ccdc6f36c9 service neutron] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:18.507997 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d14b01-2a49-4798-b78d-42ccdc6f36c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:18.508290 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d14b01-2a49-4798-b78d-42ccdc6f36c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:18.517885 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d14b01-2a49-4798-b78d-42ccdc6f36c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:18:18.519584 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d14b01-2a49-4798-b78d-42ccdc6f36c9 service neutron] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:18:18.529481 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 782/1530] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:18:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:18:20.747317 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a3b9b6-6b23-4c15-abb5-248fe3d28d11 None None] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:20.747900 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a3b9b6-6b23-4c15-abb5-248fe3d28d11 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:20.747900 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a3b9b6-6b23-4c15-abb5-248fe3d28d11 None None] PATH_INFO: `/` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:20.749392 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 749/1531] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 23 15:18:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 23 15:18:20.755280 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b01b19-dc1d-4873-b901-2ca5fbd02289 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:20.755773 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b01b19-dc1d-4873-b901-2ca5fbd02289 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:20.755880 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b01b19-dc1d-4873-b901-2ca5fbd02289 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:20.777194 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-c3b01b19-dc1d-4873-b901-2ca5fbd02289 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:18:20.780326 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-c3b01b19-dc1d-4873-b901-2ca5fbd02289 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:18:20.788615 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 783/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 23 15:18:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:18:21.083799 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a3b9b6-6b23-4c15-abb5-248fe3d28d11 None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:18:21.095507 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a3b9b6-6b23-4c15-abb5-248fe3d28d11 None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:18:21.099328 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54dc99c2-6df3-46d7-82f2-d7bbb4bada5a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd259178570864ffeb936ee10dfec6b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:18:21.099969 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dc99c2-6df3-46d7-82f2-d7bbb4bada5a service neutron] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:21.100283 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dc99c2-6df3-46d7-82f2-d7bbb4bada5a service neutron] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:21.100585 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dc99c2-6df3-46d7-82f2-d7bbb4bada5a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:21.110541 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54dc99c2-6df3-46d7-82f2-d7bbb4bada5a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:18:21.112003 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54dc99c2-6df3-46d7-82f2-d7bbb4bada5a service neutron] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:18:21.122789 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 750/1533] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:18:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:18:24.848490 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223d696e-a81a-4f19-aa13-060e58a0b122 None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:24.849074 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223d696e-a81a-4f19-aa13-060e58a0b122 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:24.849074 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223d696e-a81a-4f19-aa13-060e58a0b122 None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:24.850414 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 784/1534] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 23 15:18:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 23 15:18:24.855820 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e679df09-7fd5-4e74-a175-6d9a6f0d8711 None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:24.856312 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e679df09-7fd5-4e74-a175-6d9a6f0d8711 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:24.856412 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e679df09-7fd5-4e74-a175-6d9a6f0d8711 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:24.864001 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=58 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:18:24.877066 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-e679df09-7fd5-4e74-a175-6d9a6f0d8711 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:18:24.878370 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-e679df09-7fd5-4e74-a175-6d9a6f0d8711 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:18:24.886679 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 751/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 23 15:18:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:18:25.195626 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223d696e-a81a-4f19-aa13-060e58a0b122 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:18:25.197365 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-223d696e-a81a-4f19-aa13-060e58a0b122 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:18:25.257351 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223d696e-a81a-4f19-aa13-060e58a0b122 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:18:25.261262 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77278d2a-c1a5-4702-97a7-79e0f75ca584 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feca6278be904ef8b0dc580a67e477f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:18:25.261887 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77278d2a-c1a5-4702-97a7-79e0f75ca584 service cinder] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:25.262240 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77278d2a-c1a5-4702-97a7-79e0f75ca584 service cinder] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:25.262518 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77278d2a-c1a5-4702-97a7-79e0f75ca584 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:25.273137 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77278d2a-c1a5-4702-97a7-79e0f75ca584 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:18:25.274614 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77278d2a-c1a5-4702-97a7-79e0f75ca584 service cinder] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:18:25.285445 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 785/1536] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:18:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:18:27.519495 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a35ebf-b51c-4d39-97fc-6ea2b6302fc0 None None] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:27.520176 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a35ebf-b51c-4d39-97fc-6ea2b6302fc0 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:27.520176 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a35ebf-b51c-4d39-97fc-6ea2b6302fc0 None None] PATH_INFO: `/` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:27.521563 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 752/1537] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 23 15:18:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:18:27.527425 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896124c3-0fdd-48b6-9eb3-868cad5055ef None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:27.528000 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896124c3-0fdd-48b6-9eb3-868cad5055ef None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:27.528120 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896124c3-0fdd-48b6-9eb3-868cad5055ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:27.549152 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-896124c3-0fdd-48b6-9eb3-868cad5055ef None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:18:27.550389 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-896124c3-0fdd-48b6-9eb3-868cad5055ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:18:27.558653 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 786/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 23 15:18:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:18:27.864711 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a35ebf-b51c-4d39-97fc-6ea2b6302fc0 None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:18:27.866402 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-88a35ebf-b51c-4d39-97fc-6ea2b6302fc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:18:27.879538 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a35ebf-b51c-4d39-97fc-6ea2b6302fc0 None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:18:27.884946 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5364c24f-1e9d-4403-92be-43942873af5b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feca6278be904ef8b0dc580a67e477f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:18:27.885663 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5364c24f-1e9d-4403-92be-43942873af5b service cinder] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:27.886063 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5364c24f-1e9d-4403-92be-43942873af5b service cinder] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:27.886467 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5364c24f-1e9d-4403-92be-43942873af5b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:27.896529 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5364c24f-1e9d-4403-92be-43942873af5b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:18:27.897983 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5364c24f-1e9d-4403-92be-43942873af5b service cinder] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:18:27.908342 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 753/1539] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:18:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:18:37.543063 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=63 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:18:37.903318 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=8 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:19:55.201729 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da20b4eb-c375-4da5-9cb5-922f33e3bcdc None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:55.203231 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da20b4eb-c375-4da5-9cb5-922f33e3bcdc None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:55.203231 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da20b4eb-c375-4da5-9cb5-922f33e3bcdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:55.233805 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-da20b4eb-c375-4da5-9cb5-922f33e3bcdc None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:19:55.235441 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-da20b4eb-c375-4da5-9cb5-922f33e3bcdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:19:55.246457 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 787/1540] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:19:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:19:55.256013 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3881c2f3-8e77-47b0-8df9-f54e190d2d3b None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:55.256626 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3881c2f3-8e77-47b0-8df9-f54e190d2d3b None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:55.256626 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3881c2f3-8e77-47b0-8df9-f54e190d2d3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:55.277852 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7519a545-fd5b-440f-b929-d091be791ddd None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:55.278207 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7519a545-fd5b-440f-b929-d091be791ddd None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:55.278412 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7519a545-fd5b-440f-b929-d091be791ddd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:55.293356 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-3881c2f3-8e77-47b0-8df9-f54e190d2d3b None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:19:55.294607 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-3881c2f3-8e77-47b0-8df9-f54e190d2d3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:19:55.303138 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-7519a545-fd5b-440f-b929-d091be791ddd None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:19:55.308800 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-7519a545-fd5b-440f-b929-d091be791ddd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:19:55.314312 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 754/1541] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:19:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:19:55.322837 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 788/1542] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:19:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:19:55.324710 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3881c2f3-8e77-47b0-8df9-f54e190d2d3b None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:19:55.333862 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4391bb7-7164-4c55-98ad-903285e682f9 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:55.334051 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4391bb7-7164-4c55-98ad-903285e682f9 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:55.334325 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4391bb7-7164-4c55-98ad-903285e682f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:55.341146 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3881c2f3-8e77-47b0-8df9-f54e190d2d3b None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:19:55.355060 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106e9f83-f96d-4c07-882a-f73cb6caed3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:19:55.355060 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106e9f83-f96d-4c07-882a-f73cb6caed3b admin admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:55.355060 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106e9f83-f96d-4c07-882a-f73cb6caed3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:55.355060 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106e9f83-f96d-4c07-882a-f73cb6caed3b admin admin] PATH_INFO: `/v3/domains` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:55.366945 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106e9f83-f96d-4c07-882a-f73cb6caed3b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:19:55.366945 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106e9f83-f96d-4c07-882a-f73cb6caed3b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:19:55.366945 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106e9f83-f96d-4c07-882a-f73cb6caed3b admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:19:55.377053 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-e4391bb7-7164-4c55-98ad-903285e682f9 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:19:55.378300 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-e4391bb7-7164-4c55-98ad-903285e682f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:19:55.383941 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 755/1543] 10.222.0.55 () {58 vars in 1153 bytes} [Fri Sep 23 15:19:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:19:55.385477 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22df76bd-b5a4-4910-939e-e3d933562306 None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:55.385767 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22df76bd-b5a4-4910-939e-e3d933562306 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:55.386045 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22df76bd-b5a4-4910-939e-e3d933562306 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:55.397932 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 789/1544] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:19:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:19:55.398899 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4391bb7-7164-4c55-98ad-903285e682f9 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:19:55.416597 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4391bb7-7164-4c55-98ad-903285e682f9 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:19:55.425616 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-22df76bd-b5a4-4910-939e-e3d933562306 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:19:55.426884 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-22df76bd-b5a4-4910-939e-e3d933562306 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:19:55.438708 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb08b27f-7797-4861-9434-5a9d1bdf5cc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:19:55.439359 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb08b27f-7797-4861-9434-5a9d1bdf5cc6 admin admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:55.439644 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb08b27f-7797-4861-9434-5a9d1bdf5cc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:55.439920 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb08b27f-7797-4861-9434-5a9d1bdf5cc6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:55.440593 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb08b27f-7797-4861-9434-5a9d1bdf5cc6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:19:55.442807 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 756/1545] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:19:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:19:55.442950 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb08b27f-7797-4861-9434-5a9d1bdf5cc6 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:19:55.443423 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22df76bd-b5a4-4910-939e-e3d933562306 None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:19:55.459134 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22df76bd-b5a4-4910-939e-e3d933562306 None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:19:55.477588 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9177b08-8be5-49af-b13c-abe807a7378d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:19:55.481376 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9177b08-8be5-49af-b13c-abe807a7378d admin admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:55.481376 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9177b08-8be5-49af-b13c-abe807a7378d admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:55.481376 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9177b08-8be5-49af-b13c-abe807a7378d admin admin] PATH_INFO: `/v3/domains` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:55.481376 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9177b08-8be5-49af-b13c-abe807a7378d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:19:55.481376 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9177b08-8be5-49af-b13c-abe807a7378d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:19:55.481376 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9177b08-8be5-49af-b13c-abe807a7378d admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:19:55.500542 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 757/1546] 10.222.0.55 () {58 vars in 1153 bytes} [Fri Sep 23 15:19:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:19:55.501951 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cc4697-2015-4c6f-b759-a282ae0e41c3 None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:55.502281 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cc4697-2015-4c6f-b759-a282ae0e41c3 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:55.502696 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cc4697-2015-4c6f-b759-a282ae0e41c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:55.539535 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-80cc4697-2015-4c6f-b759-a282ae0e41c3 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:19:55.540823 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-80cc4697-2015-4c6f-b759-a282ae0e41c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:19:55.553356 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 758/1547] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:19:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:19:55.554651 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16b4a74-d189-497d-a963-d4ca9864efeb None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:55.554942 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16b4a74-d189-497d-a963-d4ca9864efeb None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:55.555221 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16b4a74-d189-497d-a963-d4ca9864efeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:55.581723 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-e16b4a74-d189-497d-a963-d4ca9864efeb None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:19:55.582920 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-e16b4a74-d189-497d-a963-d4ca9864efeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:19:55.591030 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 759/1548] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:19:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:19:55.591814 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16b4a74-d189-497d-a963-d4ca9864efeb None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:19:55.602630 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16b4a74-d189-497d-a963-d4ca9864efeb None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:19:55.609098 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5866e47-e302-460b-9948-ef6eaf977173 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:19:55.609504 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5866e47-e302-460b-9948-ef6eaf977173 admin admin] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:55.610102 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5866e47-e302-460b-9948-ef6eaf977173 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:55.610445 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5866e47-e302-460b-9948-ef6eaf977173 admin admin] PATH_INFO: `/v3/projects` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:55.611274 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5866e47-e302-460b-9948-ef6eaf977173 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:19:55.612586 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: WARNING py.warnings [None req-a5866e47-e302-460b-9948-ef6eaf977173 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 23 15:19:55.612586 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: warnings.warn(msg) Sep 23 15:19:55.612586 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: Sep 23 15:19:55.614283 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5866e47-e302-460b-9948-ef6eaf977173 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:19:55.707936 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 790/1549] 10.222.0.55 () {60 vars in 1153 bytes} [Fri Sep 23 15:19:55 2022] POST /identity/v3/projects => generated 377 bytes in 309 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:19:55.709448 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569dff63-5aae-4e70-95f9-ba2288c76e81 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:55.709734 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569dff63-5aae-4e70-95f9-ba2288c76e81 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:55.710016 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569dff63-5aae-4e70-95f9-ba2288c76e81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:55.730026 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-569dff63-5aae-4e70-95f9-ba2288c76e81 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:19:55.731244 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-569dff63-5aae-4e70-95f9-ba2288c76e81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:19:55.760605 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 760/1550] 10.222.0.55 () {60 vars in 1153 bytes} [Fri Sep 23 15:19:55 2022] POST /identity/v3/projects => generated 383 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:19:55.768414 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5866e47-e302-460b-9948-ef6eaf977173 admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:19:55.809451 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 791/1551] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:19:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:19:55.810781 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3298745-6ed0-4da8-96b0-af7939d9d7c1 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:55.811073 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3298745-6ed0-4da8-96b0-af7939d9d7c1 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:55.811409 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3298745-6ed0-4da8-96b0-af7939d9d7c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:55.833166 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-a3298745-6ed0-4da8-96b0-af7939d9d7c1 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:19:55.834403 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-a3298745-6ed0-4da8-96b0-af7939d9d7c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:19:55.839767 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5866e47-e302-460b-9948-ef6eaf977173 admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:19:55.844058 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 792/1552] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:19:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:19:55.845213 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12030406-4843-4628-8b89-e0881041f47e None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:55.845504 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12030406-4843-4628-8b89-e0881041f47e None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:55.845777 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12030406-4843-4628-8b89-e0881041f47e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:55.849587 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3673c44a-5c29-4721-a326-ea607c7e51dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:19:55.850196 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3673c44a-5c29-4721-a326-ea607c7e51dc admin admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:55.850551 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3673c44a-5c29-4721-a326-ea607c7e51dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:55.850968 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3673c44a-5c29-4721-a326-ea607c7e51dc admin admin] PATH_INFO: `/v3/domains` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:55.851676 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3673c44a-5c29-4721-a326-ea607c7e51dc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:19:55.851966 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3673c44a-5c29-4721-a326-ea607c7e51dc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:19:55.853722 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3673c44a-5c29-4721-a326-ea607c7e51dc admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:19:55.875400 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 761/1553] 10.222.0.55 () {58 vars in 1153 bytes} [Fri Sep 23 15:19:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:19:55.875640 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-12030406-4843-4628-8b89-e0881041f47e None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:19:55.875794 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb93a74-bc3a-4fa0-ab9a-e49d33513e72 None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:55.875904 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-12030406-4843-4628-8b89-e0881041f47e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:19:55.876034 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb93a74-bc3a-4fa0-ab9a-e49d33513e72 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:55.876034 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb93a74-bc3a-4fa0-ab9a-e49d33513e72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:55.876231 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 793/1554] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:19:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:19:55.880105 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8d2f5f-53fd-4f74-9057-8c1f444ef638 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:55.880105 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8d2f5f-53fd-4f74-9057-8c1f444ef638 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:55.880105 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8d2f5f-53fd-4f74-9057-8c1f444ef638 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:55.888691 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-8bb93a74-bc3a-4fa0-ab9a-e49d33513e72 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:19:55.891268 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-8bb93a74-bc3a-4fa0-ab9a-e49d33513e72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:19:55.898494 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-ab8d2f5f-53fd-4f74-9057-8c1f444ef638 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:19:55.899677 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-ab8d2f5f-53fd-4f74-9057-8c1f444ef638 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:19:55.899818 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 762/1555] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:19:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:19:55.900544 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb93a74-bc3a-4fa0-ab9a-e49d33513e72 None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:19:55.908320 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 794/1556] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:19:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:19:55.909161 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8d2f5f-53fd-4f74-9057-8c1f444ef638 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:19:55.915194 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb93a74-bc3a-4fa0-ab9a-e49d33513e72 None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:19:55.920488 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd53f11f-9612-445a-adff-cdecea998210 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:19:55.920934 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8d2f5f-53fd-4f74-9057-8c1f444ef638 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:19:55.921596 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd53f11f-9612-445a-adff-cdecea998210 admin admin] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:55.921596 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd53f11f-9612-445a-adff-cdecea998210 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:55.921886 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd53f11f-9612-445a-adff-cdecea998210 admin admin] PATH_INFO: `/v3/users` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:55.922312 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd53f11f-9612-445a-adff-cdecea998210 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:19:55.924262 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd53f11f-9612-445a-adff-cdecea998210 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:19:55.925636 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b1784b-a58f-450d-8601-6e2bfa10695f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:19:55.926263 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b1784b-a58f-450d-8601-6e2bfa10695f admin admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:55.926540 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b1784b-a58f-450d-8601-6e2bfa10695f admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:55.926812 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b1784b-a58f-450d-8601-6e2bfa10695f admin admin] PATH_INFO: `/v3/users` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:55.927431 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b1784b-a58f-450d-8601-6e2bfa10695f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:19:55.928457 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: WARNING py.warnings [None req-54b1784b-a58f-450d-8601-6e2bfa10695f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 23 15:19:55.928457 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: warnings.warn(msg) Sep 23 15:19:55.928457 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: Sep 23 15:19:55.929640 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b1784b-a58f-450d-8601-6e2bfa10695f admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:19:56.085101 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 763/1557] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Sep 23 15:19:55 2022] POST /identity/v3/users => generated 289 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:19:56.086607 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd4429d-17c2-4670-88a2-a03bb55d68ee None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:56.086891 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd4429d-17c2-4670-88a2-a03bb55d68ee None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:56.087162 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd4429d-17c2-4670-88a2-a03bb55d68ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:56.101525 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 795/1558] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Sep 23 15:19:55 2022] POST /identity/v3/users => generated 292 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:19:56.102803 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a7914e-99b1-4888-b081-3aed58c37258 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:56.103089 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a7914e-99b1-4888-b081-3aed58c37258 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:56.103368 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a7914e-99b1-4888-b081-3aed58c37258 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:56.110904 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-fdd4429d-17c2-4670-88a2-a03bb55d68ee None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:19:56.112156 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-fdd4429d-17c2-4670-88a2-a03bb55d68ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:19:56.119995 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 764/1559] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:19:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:19:56.120849 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd4429d-17c2-4670-88a2-a03bb55d68ee None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:19:56.124803 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-92a7914e-99b1-4888-b081-3aed58c37258 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:19:56.125822 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-92a7914e-99b1-4888-b081-3aed58c37258 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:19:56.130733 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd4429d-17c2-4670-88a2-a03bb55d68ee None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:19:56.134453 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 796/1560] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:19:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:19:56.135696 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa4b1ba-4105-4a26-9ad7-297426396895 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:19:56.135857 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee80a98-bb3f-4005-8fa2-5eae94c5296b None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:56.136504 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee80a98-bb3f-4005-8fa2-5eae94c5296b None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:56.136504 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee80a98-bb3f-4005-8fa2-5eae94c5296b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:56.136724 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa4b1ba-4105-4a26-9ad7-297426396895 admin admin] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:56.136970 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa4b1ba-4105-4a26-9ad7-297426396895 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:56.137520 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa4b1ba-4105-4a26-9ad7-297426396895 admin admin] PATH_INFO: `/v3/projects` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:56.138452 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa4b1ba-4105-4a26-9ad7-297426396895 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:19:56.141108 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa4b1ba-4105-4a26-9ad7-297426396895 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:19:56.158477 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-4ee80a98-bb3f-4005-8fa2-5eae94c5296b None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:19:56.159709 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-4ee80a98-bb3f-4005-8fa2-5eae94c5296b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:19:56.167784 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 797/1561] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:19:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:19:56.168605 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee80a98-bb3f-4005-8fa2-5eae94c5296b None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:19:56.178105 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee80a98-bb3f-4005-8fa2-5eae94c5296b None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:19:56.182733 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9402e863-1448-4e93-bf99-0dc0ef2f5271 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:19:56.183325 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9402e863-1448-4e93-bf99-0dc0ef2f5271 admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:56.183608 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9402e863-1448-4e93-bf99-0dc0ef2f5271 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:56.183884 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9402e863-1448-4e93-bf99-0dc0ef2f5271 admin admin] PATH_INFO: `/v3/domains` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:56.184570 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9402e863-1448-4e93-bf99-0dc0ef2f5271 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:19:56.184877 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9402e863-1448-4e93-bf99-0dc0ef2f5271 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:19:56.185886 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: WARNING py.warnings [None req-9402e863-1448-4e93-bf99-0dc0ef2f5271 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 23 15:19:56.185886 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: warnings.warn(msg) Sep 23 15:19:56.185886 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: Sep 23 15:19:56.187417 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9402e863-1448-4e93-bf99-0dc0ef2f5271 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:19:56.205366 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 798/1562] 10.222.0.55 () {58 vars in 1153 bytes} [Fri Sep 23 15:19:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:19:56.205366 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbb47a0-5a7d-497c-a42c-09f246e3c935 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:56.205366 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbb47a0-5a7d-497c-a42c-09f246e3c935 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:56.205366 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbb47a0-5a7d-497c-a42c-09f246e3c935 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:56.222546 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-fcbb47a0-5a7d-497c-a42c-09f246e3c935 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:19:56.223741 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-fcbb47a0-5a7d-497c-a42c-09f246e3c935 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:19:56.231663 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 799/1563] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:19:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:19:56.232457 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbb47a0-5a7d-497c-a42c-09f246e3c935 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:19:56.241698 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbb47a0-5a7d-497c-a42c-09f246e3c935 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:19:56.246231 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f7dcae-240d-48a0-a038-191cb1bb3ea5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:19:56.246809 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f7dcae-240d-48a0-a038-191cb1bb3ea5 admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:56.247084 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f7dcae-240d-48a0-a038-191cb1bb3ea5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:56.247350 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f7dcae-240d-48a0-a038-191cb1bb3ea5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:56.247969 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f7dcae-240d-48a0-a038-191cb1bb3ea5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:19:56.250278 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f7dcae-240d-48a0-a038-191cb1bb3ea5 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:19:56.260707 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 800/1564] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Sep 23 15:19:56 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:19:56.264031 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f7dcae-240d-48a0-a038-191cb1bb3ea5 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:19:56.272199 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f7dcae-240d-48a0-a038-191cb1bb3ea5 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:19:56.276781 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44a19ca-09ce-4c0d-9dd0-4bbb928a2ac4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:19:56.277362 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44a19ca-09ce-4c0d-9dd0-4bbb928a2ac4 admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:56.277645 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44a19ca-09ce-4c0d-9dd0-4bbb928a2ac4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:56.277917 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44a19ca-09ce-4c0d-9dd0-4bbb928a2ac4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:56.278538 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44a19ca-09ce-4c0d-9dd0-4bbb928a2ac4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:19:56.280355 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44a19ca-09ce-4c0d-9dd0-4bbb928a2ac4 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:19:56.290630 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 801/1565] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Sep 23 15:19:56 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:19:56.291457 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44a19ca-09ce-4c0d-9dd0-4bbb928a2ac4 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:19:56.301243 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44a19ca-09ce-4c0d-9dd0-4bbb928a2ac4 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:19:56.306025 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7577f9fb-26f6-4162-9efa-6e2232f7fe13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:19:56.306615 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7577f9fb-26f6-4162-9efa-6e2232f7fe13 admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:56.306898 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7577f9fb-26f6-4162-9efa-6e2232f7fe13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:56.307173 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7577f9fb-26f6-4162-9efa-6e2232f7fe13 admin admin] PATH_INFO: `/v3/domains` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:56.307820 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7577f9fb-26f6-4162-9efa-6e2232f7fe13 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:19:56.308154 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7577f9fb-26f6-4162-9efa-6e2232f7fe13 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:19:56.309753 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7577f9fb-26f6-4162-9efa-6e2232f7fe13 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:19:56.322628 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 802/1566] 10.222.0.55 () {58 vars in 1153 bytes} [Fri Sep 23 15:19:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:19:56.323453 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7577f9fb-26f6-4162-9efa-6e2232f7fe13 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:19:56.334142 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7577f9fb-26f6-4162-9efa-6e2232f7fe13 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:19:56.350209 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c471a2-6e0e-4b0a-a738-7a5d1587df01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:19:56.350209 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c471a2-6e0e-4b0a-a738-7a5d1587df01 admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:56.350209 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c471a2-6e0e-4b0a-a738-7a5d1587df01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:56.350209 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c471a2-6e0e-4b0a-a738-7a5d1587df01 admin admin] PATH_INFO: `/v3/domains` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:56.350642 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c471a2-6e0e-4b0a-a738-7a5d1587df01 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:19:56.350642 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c471a2-6e0e-4b0a-a738-7a5d1587df01 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:19:56.350800 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 765/1567] 10.222.0.55 () {60 vars in 1153 bytes} [Fri Sep 23 15:19:56 2022] POST /identity/v3/projects => generated 391 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:19:56.351310 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c471a2-6e0e-4b0a-a738-7a5d1587df01 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:19:56.351423 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa4b1ba-4105-4a26-9ad7-297426396895 admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:19:56.352179 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:19:56.356581 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 803/1568] 10.222.0.55 () {58 vars in 1153 bytes} [Fri Sep 23 15:19:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:19:56.357444 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c471a2-6e0e-4b0a-a738-7a5d1587df01 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:19:56.368136 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c471a2-6e0e-4b0a-a738-7a5d1587df01 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:19:56.390345 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:19:56.422812 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa4b1ba-4105-4a26-9ad7-297426396895 admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:19:56.427578 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a33a7ee-6a67-432b-bed3-1ba6b55e197e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:19:56.445996 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a33a7ee-6a67-432b-bed3-1ba6b55e197e admin admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:56.445996 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a33a7ee-6a67-432b-bed3-1ba6b55e197e admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:56.445996 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a33a7ee-6a67-432b-bed3-1ba6b55e197e admin admin] PATH_INFO: `/v3/domains` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:56.445996 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a33a7ee-6a67-432b-bed3-1ba6b55e197e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:19:56.445996 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a33a7ee-6a67-432b-bed3-1ba6b55e197e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:19:56.445996 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a33a7ee-6a67-432b-bed3-1ba6b55e197e admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:19:56.446876 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8eb4d6-e3fd-4cac-a314-28942a07fe44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:19:56.446876 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8eb4d6-e3fd-4cac-a314-28942a07fe44 admin admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:56.446876 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8eb4d6-e3fd-4cac-a314-28942a07fe44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:56.446876 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8eb4d6-e3fd-4cac-a314-28942a07fe44 admin admin] PATH_INFO: `/v3/projects` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:56.447521 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8eb4d6-e3fd-4cac-a314-28942a07fe44 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:19:56.447521 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8eb4d6-e3fd-4cac-a314-28942a07fe44 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:19:56.447798 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 766/1569] 10.222.0.55 () {58 vars in 1153 bytes} [Fri Sep 23 15:19:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:19:56.447798 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a33a7ee-6a67-432b-bed3-1ba6b55e197e admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:19:56.456389 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a33a7ee-6a67-432b-bed3-1ba6b55e197e admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:19:56.461867 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450b011b-9a3f-447a-bf67-826d68d10dbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:19:56.462515 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450b011b-9a3f-447a-bf67-826d68d10dbf admin admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:56.462828 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450b011b-9a3f-447a-bf67-826d68d10dbf admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:56.463134 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450b011b-9a3f-447a-bf67-826d68d10dbf admin admin] PATH_INFO: `/v3/domains` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:56.463843 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450b011b-9a3f-447a-bf67-826d68d10dbf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:19:56.464245 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450b011b-9a3f-447a-bf67-826d68d10dbf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:19:56.465895 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450b011b-9a3f-447a-bf67-826d68d10dbf admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:19:56.479162 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 767/1570] 10.222.0.55 () {58 vars in 1153 bytes} [Fri Sep 23 15:19:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:19:56.480051 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450b011b-9a3f-447a-bf67-826d68d10dbf admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:19:56.490656 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450b011b-9a3f-447a-bf67-826d68d10dbf admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:19:56.495764 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5fcaac-31b6-4017-a226-b9e2d2cb1b6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:19:56.496432 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5fcaac-31b6-4017-a226-b9e2d2cb1b6e admin admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:56.496714 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5fcaac-31b6-4017-a226-b9e2d2cb1b6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:56.496986 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5fcaac-31b6-4017-a226-b9e2d2cb1b6e admin admin] PATH_INFO: `/v3/roles` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:56.497631 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5fcaac-31b6-4017-a226-b9e2d2cb1b6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:19:56.499834 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5fcaac-31b6-4017-a226-b9e2d2cb1b6e admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:19:56.510072 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 768/1571] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Sep 23 15:19:56 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:19:56.510900 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5fcaac-31b6-4017-a226-b9e2d2cb1b6e admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:19:56.521590 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5fcaac-31b6-4017-a226-b9e2d2cb1b6e admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:19:56.526167 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c92f17-2ad2-427e-ab57-6c1e5d188937 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:19:56.526788 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c92f17-2ad2-427e-ab57-6c1e5d188937 admin admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:56.527066 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c92f17-2ad2-427e-ab57-6c1e5d188937 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:56.527344 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c92f17-2ad2-427e-ab57-6c1e5d188937 admin admin] PATH_INFO: `/v3/roles` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:56.527968 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c92f17-2ad2-427e-ab57-6c1e5d188937 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:19:56.529584 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c92f17-2ad2-427e-ab57-6c1e5d188937 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:19:56.539513 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 769/1572] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Sep 23 15:19:56 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:19:56.540329 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c92f17-2ad2-427e-ab57-6c1e5d188937 admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:19:56.550369 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c92f17-2ad2-427e-ab57-6c1e5d188937 admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:19:56.555032 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d41f3e3-d7a6-4440-98e8-a19d3bd5f2a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:19:56.555626 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d41f3e3-d7a6-4440-98e8-a19d3bd5f2a4 admin admin] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:56.555905 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d41f3e3-d7a6-4440-98e8-a19d3bd5f2a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:56.556203 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d41f3e3-d7a6-4440-98e8-a19d3bd5f2a4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:56.556833 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d41f3e3-d7a6-4440-98e8-a19d3bd5f2a4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:19:56.558725 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d41f3e3-d7a6-4440-98e8-a19d3bd5f2a4 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:19:56.583741 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 804/1573] 10.222.0.55 () {60 vars in 1153 bytes} [Fri Sep 23 15:19:56 2022] POST /identity/v3/projects => generated 373 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:19:56.584079 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8eb4d6-e3fd-4cac-a314-28942a07fe44 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:19:56.653018 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8eb4d6-e3fd-4cac-a314-28942a07fe44 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:19:56.653801 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4881188-0b4e-48c6-823f-f796e1d4e6b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:19:56.654449 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4881188-0b4e-48c6-823f-f796e1d4e6b8 admin admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:56.654728 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4881188-0b4e-48c6-823f-f796e1d4e6b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:56.655001 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4881188-0b4e-48c6-823f-f796e1d4e6b8 admin admin] PATH_INFO: `/v3/users` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:56.656026 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4881188-0b4e-48c6-823f-f796e1d4e6b8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:19:56.658107 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4881188-0b4e-48c6-823f-f796e1d4e6b8 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:19:56.680322 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 770/1574] 10.222.0.55 () {60 vars in 1153 bytes} [Fri Sep 23 15:19:56 2022] POST /identity/v3/projects => generated 387 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:19:56.681406 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d41f3e3-d7a6-4440-98e8-a19d3bd5f2a4 admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:19:56.750912 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d41f3e3-d7a6-4440-98e8-a19d3bd5f2a4 admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:19:56.755843 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28cda53-d1c9-48c8-859f-343875d030f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:19:56.756531 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28cda53-d1c9-48c8-859f-343875d030f9 admin admin] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:56.756817 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28cda53-d1c9-48c8-859f-343875d030f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:56.757373 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28cda53-d1c9-48c8-859f-343875d030f9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:56.757743 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28cda53-d1c9-48c8-859f-343875d030f9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:19:56.759802 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28cda53-d1c9-48c8-859f-343875d030f9 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:19:56.825823 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 805/1575] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Sep 23 15:19:56 2022] POST /identity/v3/users => generated 296 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:19:56.826667 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4881188-0b4e-48c6-823f-f796e1d4e6b8 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:19:56.838421 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4881188-0b4e-48c6-823f-f796e1d4e6b8 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:19:56.845838 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c7b896-213e-4bd1-963a-e9a5f4734932 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:19:56.846484 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c7b896-213e-4bd1-963a-e9a5f4734932 admin admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:56.846767 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c7b896-213e-4bd1-963a-e9a5f4734932 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:56.847044 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c7b896-213e-4bd1-963a-e9a5f4734932 admin admin] PATH_INFO: `/v3/projects` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:56.847820 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c7b896-213e-4bd1-963a-e9a5f4734932 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:19:56.849799 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c7b896-213e-4bd1-963a-e9a5f4734932 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:19:56.932878 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 771/1576] 10.222.0.55 () {60 vars in 1153 bytes} [Fri Sep 23 15:19:56 2022] POST /identity/v3/projects => generated 379 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:19:56.933643 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28cda53-d1c9-48c8-859f-343875d030f9 admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:19:56.993625 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:19:57.007814 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 806/1577] 10.222.0.55 () {60 vars in 1153 bytes} [Fri Sep 23 15:19:56 2022] POST /identity/v3/projects => generated 405 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:19:57.007814 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c7b896-213e-4bd1-963a-e9a5f4734932 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:19:57.014216 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28cda53-d1c9-48c8-859f-343875d030f9 admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:19:57.069471 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:19:57.084141 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c7b896-213e-4bd1-963a-e9a5f4734932 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:19:57.089620 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b3a13e-71c2-417d-80ca-41b4757889fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:19:57.090800 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b3a13e-71c2-417d-80ca-41b4757889fc admin admin] REQUEST_METHOD: `PUT` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:57.090919 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633e13dc-30b1-4f17-b770-915b5209fb55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:19:57.091593 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b3a13e-71c2-417d-80ca-41b4757889fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:57.091713 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633e13dc-30b1-4f17-b770-915b5209fb55 admin admin] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:57.091713 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633e13dc-30b1-4f17-b770-915b5209fb55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:57.091911 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b3a13e-71c2-417d-80ca-41b4757889fc admin admin] PATH_INFO: `/v3/projects/ab19e9f7cb0e49548a9ab74b9d2b4aec/users/f5f0ecaf38e74c96bc439fde75d93d37/roles/af257be2dee54d6b942f58650196582c` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:57.092101 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633e13dc-30b1-4f17-b770-915b5209fb55 admin admin] PATH_INFO: `/v3/projects` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:57.092699 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633e13dc-30b1-4f17-b770-915b5209fb55 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:19:57.094939 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633e13dc-30b1-4f17-b770-915b5209fb55 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:19:57.113538 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b3a13e-71c2-417d-80ca-41b4757889fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab19e9f7cb0e49548a9ab74b9d2b4aec, user_id=f5f0ecaf38e74c96bc439fde75d93d37, role_id=af257be2dee54d6b942f58650196582c)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:19:57.116509 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b3a13e-71c2-417d-80ca-41b4757889fc admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:19:57.240227 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 772/1578] 10.222.0.55 () {60 vars in 1153 bytes} [Fri Sep 23 15:19:56 2022] POST /identity/v3/projects => generated 373 bytes in 307 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:19:57.241213 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633e13dc-30b1-4f17-b770-915b5209fb55 admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:19:57.257842 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 807/1579] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Sep 23 15:19:57 2022] PUT /identity/v3/projects/ab19e9f7cb0e49548a9ab74b9d2b4aec/users/f5f0ecaf38e74c96bc439fde75d93d37/roles/af257be2dee54d6b942f58650196582c => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:19:57.258847 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b3a13e-71c2-417d-80ca-41b4757889fc admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:19:57.320078 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633e13dc-30b1-4f17-b770-915b5209fb55 admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:19:57.325889 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0027077b-ee9d-4fd0-bbde-0e6233bf8e5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:19:57.326596 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b3a13e-71c2-417d-80ca-41b4757889fc admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:19:57.327535 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0027077b-ee9d-4fd0-bbde-0e6233bf8e5d admin admin] REQUEST_METHOD: `PUT` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:57.328076 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0027077b-ee9d-4fd0-bbde-0e6233bf8e5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:57.328586 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0027077b-ee9d-4fd0-bbde-0e6233bf8e5d admin admin] PATH_INFO: `/v3/projects/f87732b5735542ec8efcd0f8d43a0e76/users/c500d775ea144aa48ba2b4db12ec3407/roles/af257be2dee54d6b942f58650196582c` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:57.332085 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd7a804-62fe-49f5-aa8c-92adf8d3809d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:19:57.332702 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd7a804-62fe-49f5-aa8c-92adf8d3809d admin admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:57.332984 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd7a804-62fe-49f5-aa8c-92adf8d3809d admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:57.333257 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd7a804-62fe-49f5-aa8c-92adf8d3809d admin admin] PATH_INFO: `/v3/users` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:57.333893 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd7a804-62fe-49f5-aa8c-92adf8d3809d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:19:57.335814 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd7a804-62fe-49f5-aa8c-92adf8d3809d admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:19:57.357078 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0027077b-ee9d-4fd0-bbde-0e6233bf8e5d admin admin] RBAC: Authorizing `identity:create_grant(project_id=f87732b5735542ec8efcd0f8d43a0e76, user_id=c500d775ea144aa48ba2b4db12ec3407, role_id=af257be2dee54d6b942f58650196582c)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:19:57.359468 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: WARNING py.warnings [None req-0027077b-ee9d-4fd0-bbde-0e6233bf8e5d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 23 15:19:57.359468 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: warnings.warn(msg) Sep 23 15:19:57.359468 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: Sep 23 15:19:57.363728 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0027077b-ee9d-4fd0-bbde-0e6233bf8e5d admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:19:57.464681 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 808/1580] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Sep 23 15:19:57 2022] POST /identity/v3/users => generated 287 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:19:57.465499 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd7a804-62fe-49f5-aa8c-92adf8d3809d admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:19:57.476123 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd7a804-62fe-49f5-aa8c-92adf8d3809d admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:19:57.480647 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be92c541-4ebd-4663-a21a-37ec03ff08a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:19:57.481225 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be92c541-4ebd-4663-a21a-37ec03ff08a6 admin admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:57.482300 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be92c541-4ebd-4663-a21a-37ec03ff08a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:57.482300 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be92c541-4ebd-4663-a21a-37ec03ff08a6 admin admin] PATH_INFO: `/v3/users` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:57.482492 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be92c541-4ebd-4663-a21a-37ec03ff08a6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:19:57.484442 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be92c541-4ebd-4663-a21a-37ec03ff08a6 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:19:57.500584 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 773/1581] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Sep 23 15:19:57 2022] PUT /identity/v3/projects/f87732b5735542ec8efcd0f8d43a0e76/users/c500d775ea144aa48ba2b4db12ec3407/roles/af257be2dee54d6b942f58650196582c => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:19:57.501621 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0027077b-ee9d-4fd0-bbde-0e6233bf8e5d admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:19:57.564127 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0027077b-ee9d-4fd0-bbde-0e6233bf8e5d admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:19:57.568793 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ab8dca-9eb8-42ff-9556-011dba27858a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:19:57.569396 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ab8dca-9eb8-42ff-9556-011dba27858a admin admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:57.569698 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ab8dca-9eb8-42ff-9556-011dba27858a admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:57.569953 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ab8dca-9eb8-42ff-9556-011dba27858a admin admin] PATH_INFO: `/v3/roles` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:57.570671 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ab8dca-9eb8-42ff-9556-011dba27858a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:19:57.572233 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ab8dca-9eb8-42ff-9556-011dba27858a admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:19:57.578098 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:19:57.582993 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 774/1582] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Sep 23 15:19:57 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:19:57.585309 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ab8dca-9eb8-42ff-9556-011dba27858a admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:19:57.596306 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ab8dca-9eb8-42ff-9556-011dba27858a admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:19:57.601247 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54cced2-afef-4fe1-9b93-48de41377599 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:19:57.601911 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54cced2-afef-4fe1-9b93-48de41377599 admin admin] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:57.602194 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54cced2-afef-4fe1-9b93-48de41377599 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:57.602475 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54cced2-afef-4fe1-9b93-48de41377599 admin admin] PATH_INFO: `/v3/users` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:57.603116 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54cced2-afef-4fe1-9b93-48de41377599 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:19:57.605146 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54cced2-afef-4fe1-9b93-48de41377599 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:19:57.610613 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 809/1583] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Sep 23 15:19:57 2022] POST /identity/v3/users => generated 294 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:19:57.611474 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be92c541-4ebd-4663-a21a-37ec03ff08a6 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:19:57.622572 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be92c541-4ebd-4663-a21a-37ec03ff08a6 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:19:57.628115 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a894277-c4aa-4804-a7d5-66a2d723d812 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:19:57.628752 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a894277-c4aa-4804-a7d5-66a2d723d812 admin admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:57.629034 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a894277-c4aa-4804-a7d5-66a2d723d812 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:57.629312 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a894277-c4aa-4804-a7d5-66a2d723d812 admin admin] PATH_INFO: `/v3/users` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:57.629952 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a894277-c4aa-4804-a7d5-66a2d723d812 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:19:57.631945 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a894277-c4aa-4804-a7d5-66a2d723d812 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:19:57.724397 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 775/1584] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Sep 23 15:19:57 2022] POST /identity/v3/users => generated 290 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:19:57.725394 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54cced2-afef-4fe1-9b93-48de41377599 admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:19:57.736163 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54cced2-afef-4fe1-9b93-48de41377599 admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:19:57.740952 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc93250f-e874-4e4f-9d0f-e1a17ba63cde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:19:57.741613 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc93250f-e874-4e4f-9d0f-e1a17ba63cde admin admin] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:57.742052 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc93250f-e874-4e4f-9d0f-e1a17ba63cde admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:57.742342 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc93250f-e874-4e4f-9d0f-e1a17ba63cde admin admin] PATH_INFO: `/v3/users` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:57.742996 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc93250f-e874-4e4f-9d0f-e1a17ba63cde admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:19:57.745086 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc93250f-e874-4e4f-9d0f-e1a17ba63cde admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:19:57.773859 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 810/1585] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Sep 23 15:19:57 2022] POST /identity/v3/users => generated 303 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:19:57.773859 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66ee4c2-640a-464e-a8dd-e5b4e6c5df2e None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:57.775027 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66ee4c2-640a-464e-a8dd-e5b4e6c5df2e None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:57.775027 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66ee4c2-640a-464e-a8dd-e5b4e6c5df2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:57.794749 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-e66ee4c2-640a-464e-a8dd-e5b4e6c5df2e None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:19:57.796318 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-e66ee4c2-640a-464e-a8dd-e5b4e6c5df2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:19:57.805162 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 811/1586] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:19:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:19:57.806055 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66ee4c2-640a-464e-a8dd-e5b4e6c5df2e None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:19:57.817449 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66ee4c2-640a-464e-a8dd-e5b4e6c5df2e None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:19:57.824393 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3fa25b-7dec-4d45-b669-f95511f758e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:19:57.825250 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3fa25b-7dec-4d45-b669-f95511f758e8 admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:57.825690 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3fa25b-7dec-4d45-b669-f95511f758e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:57.826087 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3fa25b-7dec-4d45-b669-f95511f758e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:57.826746 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3fa25b-7dec-4d45-b669-f95511f758e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:19:57.829198 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3fa25b-7dec-4d45-b669-f95511f758e8 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:19:57.840676 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 812/1587] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Sep 23 15:19:57 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:19:57.841983 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08978036-3532-4399-b78f-f9079b6ab886 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:57.842268 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08978036-3532-4399-b78f-f9079b6ab886 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:57.842795 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08978036-3532-4399-b78f-f9079b6ab886 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:57.848076 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 776/1588] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Sep 23 15:19:57 2022] POST /identity/v3/users => generated 287 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:19:57.852657 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc93250f-e874-4e4f-9d0f-e1a17ba63cde admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:19:57.862578 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc93250f-e874-4e4f-9d0f-e1a17ba63cde admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:19:57.867305 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bae06e-98f4-4620-b74d-3e43b0dfedb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:19:57.867917 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bae06e-98f4-4620-b74d-3e43b0dfedb0 admin admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:57.868230 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bae06e-98f4-4620-b74d-3e43b0dfedb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:57.868503 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bae06e-98f4-4620-b74d-3e43b0dfedb0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:57.869139 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bae06e-98f4-4620-b74d-3e43b0dfedb0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:19:57.870745 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bae06e-98f4-4620-b74d-3e43b0dfedb0 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:19:57.875828 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-08978036-3532-4399-b78f-f9079b6ab886 None None] MFA Rules not processed for user `c500d775ea144aa48ba2b4db12ec3407`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:19:57.877225 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-08978036-3532-4399-b78f-f9079b6ab886 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:19:57.882309 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 777/1589] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Sep 23 15:19:57 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:19:57.883197 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bae06e-98f4-4620-b74d-3e43b0dfedb0 admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:19:57.893883 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bae06e-98f4-4620-b74d-3e43b0dfedb0 admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:19:57.899066 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defa8863-c985-43ee-9062-bb79097d4eea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:19:57.899745 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defa8863-c985-43ee-9062-bb79097d4eea admin admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:57.900051 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defa8863-c985-43ee-9062-bb79097d4eea admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:57.900326 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defa8863-c985-43ee-9062-bb79097d4eea admin admin] PATH_INFO: `/v3/roles` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:57.900964 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defa8863-c985-43ee-9062-bb79097d4eea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:19:57.902588 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defa8863-c985-43ee-9062-bb79097d4eea admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:19:57.914168 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 778/1590] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Sep 23 15:19:57 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:19:57.914808 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defa8863-c985-43ee-9062-bb79097d4eea admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:19:57.924568 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defa8863-c985-43ee-9062-bb79097d4eea admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:19:57.929324 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213fbc0a-5329-4e15-8e1f-e5519df064df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:19:57.929922 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213fbc0a-5329-4e15-8e1f-e5519df064df admin admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:57.930197 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213fbc0a-5329-4e15-8e1f-e5519df064df admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:57.930473 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213fbc0a-5329-4e15-8e1f-e5519df064df admin admin] PATH_INFO: `/v3/roles` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:57.931146 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213fbc0a-5329-4e15-8e1f-e5519df064df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:19:57.932872 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213fbc0a-5329-4e15-8e1f-e5519df064df admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:19:57.944455 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 779/1591] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Sep 23 15:19:57 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:19:57.945113 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213fbc0a-5329-4e15-8e1f-e5519df064df admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:19:57.951143 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 813/1592] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:19:57 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:19:57.952612 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08978036-3532-4399-b78f-f9079b6ab886 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:19:57.956777 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213fbc0a-5329-4e15-8e1f-e5519df064df admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:19:57.962534 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f1bdee-377c-4e51-a3a0-ab9ae96383b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:19:57.963143 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f1bdee-377c-4e51-a3a0-ab9ae96383b2 admin admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:57.963300 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:19:57.963418 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f1bdee-377c-4e51-a3a0-ab9ae96383b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:57.963674 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f1bdee-377c-4e51-a3a0-ab9ae96383b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:57.964339 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f1bdee-377c-4e51-a3a0-ab9ae96383b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:19:57.966232 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f1bdee-377c-4e51-a3a0-ab9ae96383b2 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:19:57.976806 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 780/1593] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Sep 23 15:19:57 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:19:57.977643 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f1bdee-377c-4e51-a3a0-ab9ae96383b2 admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:19:57.988666 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f1bdee-377c-4e51-a3a0-ab9ae96383b2 admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:19:57.993410 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba948cc5-a7f5-4ae2-9507-e8852b89dee2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:19:57.994047 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba948cc5-a7f5-4ae2-9507-e8852b89dee2 admin admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:57.994340 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba948cc5-a7f5-4ae2-9507-e8852b89dee2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:57.994608 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba948cc5-a7f5-4ae2-9507-e8852b89dee2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:57.995242 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba948cc5-a7f5-4ae2-9507-e8852b89dee2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:19:57.996865 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba948cc5-a7f5-4ae2-9507-e8852b89dee2 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:19:58.008630 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 781/1594] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Sep 23 15:19:57 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:19:58.014471 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba948cc5-a7f5-4ae2-9507-e8852b89dee2 admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:19:58.014603 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08978036-3532-4399-b78f-f9079b6ab886 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:19:58.018470 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a369f9-452a-497f-98df-855c41b1f610 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd259178570864ffeb936ee10dfec6b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:19:58.019103 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a369f9-452a-497f-98df-855c41b1f610 service neutron] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:58.019381 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a369f9-452a-497f-98df-855c41b1f610 service neutron] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:58.019771 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba948cc5-a7f5-4ae2-9507-e8852b89dee2 admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:19:58.020030 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a369f9-452a-497f-98df-855c41b1f610 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:58.024312 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8b7ff4-41a5-4beb-8c85-3f1b54c6f40a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:19:58.024844 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8b7ff4-41a5-4beb-8c85-3f1b54c6f40a admin admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:58.025120 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8b7ff4-41a5-4beb-8c85-3f1b54c6f40a admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:58.025390 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8b7ff4-41a5-4beb-8c85-3f1b54c6f40a admin admin] PATH_INFO: `/v3/roles` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:58.026012 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8b7ff4-41a5-4beb-8c85-3f1b54c6f40a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:19:58.027601 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8b7ff4-41a5-4beb-8c85-3f1b54c6f40a admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:19:58.029140 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a369f9-452a-497f-98df-855c41b1f610 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:19:58.031117 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a369f9-452a-497f-98df-855c41b1f610 service neutron] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:19:58.038616 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 782/1595] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Sep 23 15:19:58 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:19:58.039401 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8b7ff4-41a5-4beb-8c85-3f1b54c6f40a admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:19:58.041894 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 814/1596] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:19:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:19:58.042539 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a369f9-452a-497f-98df-855c41b1f610 service neutron] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:19:58.049480 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8b7ff4-41a5-4beb-8c85-3f1b54c6f40a admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:19:58.052664 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a369f9-452a-497f-98df-855c41b1f610 service neutron] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:19:58.054888 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2843eed9-469e-4b8d-a992-cd3662273e6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:19:58.060477 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2843eed9-469e-4b8d-a992-cd3662273e6d admin admin] REQUEST_METHOD: `PUT` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:58.060477 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2843eed9-469e-4b8d-a992-cd3662273e6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:58.060477 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2843eed9-469e-4b8d-a992-cd3662273e6d admin admin] PATH_INFO: `/v3/projects/a079aa0c75bf44619342f53006d88ff9/users/a5514abdc8b54282901367a4f2d02856/roles/af257be2dee54d6b942f58650196582c` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:58.060740 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3485ade-f2ba-44ff-bf2c-08d5f8af1556 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:19:58.060740 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3485ade-f2ba-44ff-bf2c-08d5f8af1556 admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:58.060740 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3485ade-f2ba-44ff-bf2c-08d5f8af1556 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:58.060740 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3485ade-f2ba-44ff-bf2c-08d5f8af1556 admin admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:58.061099 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3485ade-f2ba-44ff-bf2c-08d5f8af1556 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:19:58.061358 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3485ade-f2ba-44ff-bf2c-08d5f8af1556 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:19:58.073547 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2843eed9-469e-4b8d-a992-cd3662273e6d admin admin] RBAC: Authorizing `identity:create_grant(project_id=a079aa0c75bf44619342f53006d88ff9, user_id=a5514abdc8b54282901367a4f2d02856, role_id=af257be2dee54d6b942f58650196582c)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:19:58.080577 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 815/1597] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Sep 23 15:19:58 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:19:58.080577 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3485ade-f2ba-44ff-bf2c-08d5f8af1556 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:19:58.083263 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2843eed9-469e-4b8d-a992-cd3662273e6d admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:19:58.089092 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3485ade-f2ba-44ff-bf2c-08d5f8af1556 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:19:58.094673 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85db7084-a6db-432e-b688-44a6df665c9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:19:58.095283 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85db7084-a6db-432e-b688-44a6df665c9e admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:58.095566 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85db7084-a6db-432e-b688-44a6df665c9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:58.095841 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85db7084-a6db-432e-b688-44a6df665c9e admin admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:58.096519 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85db7084-a6db-432e-b688-44a6df665c9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:19:58.098111 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85db7084-a6db-432e-b688-44a6df665c9e admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:19:58.108189 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 816/1598] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Sep 23 15:19:58 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:19:58.109770 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2cacf0-cc4c-40cc-a24e-0e22c0f1ac27 None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:58.110064 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2cacf0-cc4c-40cc-a24e-0e22c0f1ac27 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:58.110350 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2cacf0-cc4c-40cc-a24e-0e22c0f1ac27 None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:58.111807 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 817/1599] 10.222.0.55 () {54 vars in 961 bytes} [Fri Sep 23 15:19:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:19:58.112438 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2cacf0-cc4c-40cc-a24e-0e22c0f1ac27 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:19:58.124951 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2cacf0-cc4c-40cc-a24e-0e22c0f1ac27 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:19:58.129598 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dae524c-cc6a-4e32-b3d7-db62498ed115 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:19:58.130188 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dae524c-cc6a-4e32-b3d7-db62498ed115 admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:58.130468 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dae524c-cc6a-4e32-b3d7-db62498ed115 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:58.130741 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dae524c-cc6a-4e32-b3d7-db62498ed115 admin admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:58.131372 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dae524c-cc6a-4e32-b3d7-db62498ed115 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:19:58.133030 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dae524c-cc6a-4e32-b3d7-db62498ed115 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:19:58.144547 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 818/1600] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Sep 23 15:19:58 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:19:58.145332 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dae524c-cc6a-4e32-b3d7-db62498ed115 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:19:58.157563 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dae524c-cc6a-4e32-b3d7-db62498ed115 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:19:58.162281 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdd47c4-1348-4883-beed-d21bfe1e2cb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:19:58.162941 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdd47c4-1348-4883-beed-d21bfe1e2cb9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:58.163223 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdd47c4-1348-4883-beed-d21bfe1e2cb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:58.163498 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdd47c4-1348-4883-beed-d21bfe1e2cb9 admin admin] PATH_INFO: `/v3/projects/b6fdf271fa004fd796efd1eaca24340f/users/e64768b37cd54dee88b845382a203869/roles/af257be2dee54d6b942f58650196582c` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:58.178426 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdd47c4-1348-4883-beed-d21bfe1e2cb9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b6fdf271fa004fd796efd1eaca24340f, user_id=e64768b37cd54dee88b845382a203869, role_id=af257be2dee54d6b942f58650196582c)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:19:58.182255 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdd47c4-1348-4883-beed-d21bfe1e2cb9 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:19:58.187794 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 783/1601] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Sep 23 15:19:58 2022] PUT /identity/v3/projects/a079aa0c75bf44619342f53006d88ff9/users/a5514abdc8b54282901367a4f2d02856/roles/af257be2dee54d6b942f58650196582c => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:19:58.188719 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2843eed9-469e-4b8d-a992-cd3662273e6d admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:19:58.254535 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2843eed9-469e-4b8d-a992-cd3662273e6d admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:19:58.259743 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c55301-d25d-4f4f-80a7-72f15faacf4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:19:58.263872 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c55301-d25d-4f4f-80a7-72f15faacf4c admin admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:58.263872 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c55301-d25d-4f4f-80a7-72f15faacf4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:58.263872 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c55301-d25d-4f4f-80a7-72f15faacf4c admin admin] PATH_INFO: `/v3/roles` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:58.263872 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c55301-d25d-4f4f-80a7-72f15faacf4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:19:58.263872 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c55301-d25d-4f4f-80a7-72f15faacf4c admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:19:58.273815 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 784/1602] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Sep 23 15:19:58 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:19:58.274767 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c55301-d25d-4f4f-80a7-72f15faacf4c admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:19:58.284777 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c55301-d25d-4f4f-80a7-72f15faacf4c admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:19:58.289322 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7c02b0-3529-41fd-83d6-f6db8aca5cb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:19:58.289942 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7c02b0-3529-41fd-83d6-f6db8aca5cb3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:58.290222 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7c02b0-3529-41fd-83d6-f6db8aca5cb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:58.290490 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7c02b0-3529-41fd-83d6-f6db8aca5cb3 admin admin] PATH_INFO: `/v3/projects/1679242155f94d48871525425f74d1c3/users/42dc378333ae4bbdbd42ce041ba15d3f/roles/af257be2dee54d6b942f58650196582c` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:58.310855 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7c02b0-3529-41fd-83d6-f6db8aca5cb3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1679242155f94d48871525425f74d1c3, user_id=42dc378333ae4bbdbd42ce041ba15d3f, role_id=af257be2dee54d6b942f58650196582c)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:19:58.314346 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7c02b0-3529-41fd-83d6-f6db8aca5cb3 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:19:58.330433 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 819/1603] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Sep 23 15:19:58 2022] PUT /identity/v3/projects/b6fdf271fa004fd796efd1eaca24340f/users/e64768b37cd54dee88b845382a203869/roles/af257be2dee54d6b942f58650196582c => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:19:58.335319 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdd47c4-1348-4883-beed-d21bfe1e2cb9 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:19:58.400720 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 785/1604] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Sep 23 15:19:58 2022] PUT /identity/v3/projects/1679242155f94d48871525425f74d1c3/users/42dc378333ae4bbdbd42ce041ba15d3f/roles/af257be2dee54d6b942f58650196582c => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:19:58.402620 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72ce4f3-fdb6-4258-9a54-9320421581f7 None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:58.402969 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72ce4f3-fdb6-4258-9a54-9320421581f7 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:58.403305 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72ce4f3-fdb6-4258-9a54-9320421581f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:58.417614 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:19:58.421755 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdd47c4-1348-4883-beed-d21bfe1e2cb9 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:19:58.447771 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-f72ce4f3-fdb6-4258-9a54-9320421581f7 None None] MFA Rules not processed for user `b3c6c7a1325a47be924268e1aa1a7a18`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:19:58.449095 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-f72ce4f3-fdb6-4258-9a54-9320421581f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:19:58.479054 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:19:58.487239 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88268bb-08af-4f88-88e5-f0226936c0c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:19:58.491805 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88268bb-08af-4f88-88e5-f0226936c0c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:58.491805 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88268bb-08af-4f88-88e5-f0226936c0c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:58.491805 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88268bb-08af-4f88-88e5-f0226936c0c5 admin admin] PATH_INFO: `/v3/projects/df328392b4f54d0c92df32df0241e95d/users/69dc5c52e36841f5aad028fc6a4b3daf/roles/af257be2dee54d6b942f58650196582c` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:58.504020 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88268bb-08af-4f88-88e5-f0226936c0c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=df328392b4f54d0c92df32df0241e95d, user_id=69dc5c52e36841f5aad028fc6a4b3daf, role_id=af257be2dee54d6b942f58650196582c)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:19:58.506992 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88268bb-08af-4f88-88e5-f0226936c0c5 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:19:58.522148 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 786/1605] 10.222.0.55 () {58 vars in 1058 bytes} [Fri Sep 23 15:19:58 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:19:58.523071 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72ce4f3-fdb6-4258-9a54-9320421581f7 None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:19:58.533690 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72ce4f3-fdb6-4258-9a54-9320421581f7 None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:19:58.538165 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1e21ff-f5f2-4695-9ae3-d22f1b7badce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:19:58.538809 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1e21ff-f5f2-4695-9ae3-d22f1b7badce admin admin] REQUEST_METHOD: `PUT` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:58.539154 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1e21ff-f5f2-4695-9ae3-d22f1b7badce admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:58.539414 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1e21ff-f5f2-4695-9ae3-d22f1b7badce admin admin] PATH_INFO: `/v3/projects/14696ebba19a41fcbd4d5fcda48bec1e/users/5c802d31a1304331bf82f4904956dbc4/roles/af257be2dee54d6b942f58650196582c` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:58.554890 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1e21ff-f5f2-4695-9ae3-d22f1b7badce admin admin] RBAC: Authorizing `identity:create_grant(project_id=14696ebba19a41fcbd4d5fcda48bec1e, user_id=5c802d31a1304331bf82f4904956dbc4, role_id=af257be2dee54d6b942f58650196582c)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:19:58.557672 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1e21ff-f5f2-4695-9ae3-d22f1b7badce admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:19:58.638946 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 820/1606] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Sep 23 15:19:58 2022] PUT /identity/v3/projects/df328392b4f54d0c92df32df0241e95d/users/69dc5c52e36841f5aad028fc6a4b3daf/roles/af257be2dee54d6b942f58650196582c => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:19:58.643214 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b70335-347b-41d6-ab71-2560ba5d578b None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:58.643214 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b70335-347b-41d6-ab71-2560ba5d578b None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:58.643214 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b70335-347b-41d6-ab71-2560ba5d578b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:58.660736 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-89b70335-347b-41d6-ab71-2560ba5d578b None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:19:58.661942 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-89b70335-347b-41d6-ab71-2560ba5d578b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:19:58.702281 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 787/1607] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Sep 23 15:19:58 2022] PUT /identity/v3/projects/14696ebba19a41fcbd4d5fcda48bec1e/users/5c802d31a1304331bf82f4904956dbc4/roles/af257be2dee54d6b942f58650196582c => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:19:58.706673 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1e21ff-f5f2-4695-9ae3-d22f1b7badce admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:19:58.732319 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 821/1608] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:19:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:19:58.732319 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e040832-5b53-4b55-aef0-f63555260748 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:58.732319 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e040832-5b53-4b55-aef0-f63555260748 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:58.732319 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e040832-5b53-4b55-aef0-f63555260748 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:58.763704 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-9e040832-5b53-4b55-aef0-f63555260748 None None] MFA Rules not processed for user `e64768b37cd54dee88b845382a203869`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:19:58.765114 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-9e040832-5b53-4b55-aef0-f63555260748 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:19:58.782307 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1e21ff-f5f2-4695-9ae3-d22f1b7badce admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:19:58.787491 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f73cd5-9f22-4302-be71-42378d14ded6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:19:58.788123 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f73cd5-9f22-4302-be71-42378d14ded6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:58.788402 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f73cd5-9f22-4302-be71-42378d14ded6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:58.788671 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f73cd5-9f22-4302-be71-42378d14ded6 admin admin] PATH_INFO: `/v3/projects/700da9a5171446dfb55095c96caa714f/users/473a152c83444c87aea51c0c1569b249/roles/af257be2dee54d6b942f58650196582c` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:58.803955 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f73cd5-9f22-4302-be71-42378d14ded6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=700da9a5171446dfb55095c96caa714f, user_id=473a152c83444c87aea51c0c1569b249, role_id=af257be2dee54d6b942f58650196582c)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:19:58.806666 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f73cd5-9f22-4302-be71-42378d14ded6 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:19:58.839589 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 822/1609] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:19:58 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:19:58.841610 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82c24fc-5bfe-4fdd-80a9-8ad42d68b4c0 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:58.841610 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82c24fc-5bfe-4fdd-80a9-8ad42d68b4c0 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:58.841882 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82c24fc-5bfe-4fdd-80a9-8ad42d68b4c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:58.860922 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-c82c24fc-5bfe-4fdd-80a9-8ad42d68b4c0 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:19:58.862125 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-c82c24fc-5bfe-4fdd-80a9-8ad42d68b4c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:19:58.869795 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 823/1610] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:19:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:19:58.873855 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82c24fc-5bfe-4fdd-80a9-8ad42d68b4c0 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:19:58.906650 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 788/1611] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Sep 23 15:19:58 2022] PUT /identity/v3/projects/700da9a5171446dfb55095c96caa714f/users/473a152c83444c87aea51c0c1569b249/roles/af257be2dee54d6b942f58650196582c => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:19:58.907957 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bcc75e-f07c-45ac-b87e-d1e1844b2026 None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:58.908379 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bcc75e-f07c-45ac-b87e-d1e1844b2026 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:58.908660 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bcc75e-f07c-45ac-b87e-d1e1844b2026 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:58.927603 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-c1bcc75e-f07c-45ac-b87e-d1e1844b2026 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:19:58.928831 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-c1bcc75e-f07c-45ac-b87e-d1e1844b2026 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:19:58.930022 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:19:58.944361 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82c24fc-5bfe-4fdd-80a9-8ad42d68b4c0 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:19:58.973555 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:19:58.996400 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 789/1612] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:19:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:19:58.996400 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78209ced-ae26-478b-854b-6160dbd8e79c None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:58.996400 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78209ced-ae26-478b-854b-6160dbd8e79c None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:58.996400 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78209ced-ae26-478b-854b-6160dbd8e79c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:59.015840 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-78209ced-ae26-478b-854b-6160dbd8e79c None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:19:59.017126 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-78209ced-ae26-478b-854b-6160dbd8e79c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:19:59.026767 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18137f8d-1b52-4959-a923-7cb9a8cea170 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3c6c7a1325a47be924268e1aa1a7a18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:19:59.027394 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18137f8d-1b52-4959-a923-7cb9a8cea170 service nova] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:59.027672 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18137f8d-1b52-4959-a923-7cb9a8cea170 service nova] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:59.027941 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18137f8d-1b52-4959-a923-7cb9a8cea170 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:59.029742 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 790/1613] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:19:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:19:59.031511 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78209ced-ae26-478b-854b-6160dbd8e79c None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:19:59.097703 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18137f8d-1b52-4959-a923-7cb9a8cea170 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:19:59.099468 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18137f8d-1b52-4959-a923-7cb9a8cea170 service nova] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:19:59.099816 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78209ced-ae26-478b-854b-6160dbd8e79c None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:19:59.104389 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d04c79-5a0c-4e4e-ba12-cbff009fff8e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd259178570864ffeb936ee10dfec6b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:19:59.105067 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d04c79-5a0c-4e4e-ba12-cbff009fff8e service neutron] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:59.105344 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d04c79-5a0c-4e4e-ba12-cbff009fff8e service neutron] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:59.105614 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d04c79-5a0c-4e4e-ba12-cbff009fff8e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:59.111064 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 824/1614] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:19:58 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:19:59.112104 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18137f8d-1b52-4959-a923-7cb9a8cea170 service nova] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:19:59.117216 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d04c79-5a0c-4e4e-ba12-cbff009fff8e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:19:59.118802 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d04c79-5a0c-4e4e-ba12-cbff009fff8e service neutron] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:19:59.123082 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18137f8d-1b52-4959-a923-7cb9a8cea170 service nova] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:19:59.129390 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f49d045-7bd5-4117-8b18-e92bdfea09d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:19:59.130183 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f49d045-7bd5-4117-8b18-e92bdfea09d5 admin admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:59.130812 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f49d045-7bd5-4117-8b18-e92bdfea09d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:59.131245 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f49d045-7bd5-4117-8b18-e92bdfea09d5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:59.131412 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 791/1615] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:19:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:19:59.132224 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f49d045-7bd5-4117-8b18-e92bdfea09d5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:19:59.132354 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d04c79-5a0c-4e4e-ba12-cbff009fff8e service neutron] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:19:59.135089 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f49d045-7bd5-4117-8b18-e92bdfea09d5 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:19:59.141768 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d04c79-5a0c-4e4e-ba12-cbff009fff8e service neutron] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:19:59.146254 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c5748c-fe55-45e5-860e-69893d32426e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd259178570864ffeb936ee10dfec6b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:19:59.146864 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c5748c-fe55-45e5-860e-69893d32426e service neutron] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:59.147136 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c5748c-fe55-45e5-860e-69893d32426e service neutron] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:59.147400 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c5748c-fe55-45e5-860e-69893d32426e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:59.158067 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c5748c-fe55-45e5-860e-69893d32426e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:19:59.160546 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c5748c-fe55-45e5-860e-69893d32426e service neutron] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:19:59.169900 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 792/1616] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:19:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:19:59.171757 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8893033d-643b-4e55-8ea6-c66a05f844d9 None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:59.172102 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8893033d-643b-4e55-8ea6-c66a05f844d9 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:59.172375 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8893033d-643b-4e55-8ea6-c66a05f844d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:59.203658 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-8893033d-643b-4e55-8ea6-c66a05f844d9 None None] MFA Rules not processed for user `473a152c83444c87aea51c0c1569b249`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:19:59.204956 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-8893033d-643b-4e55-8ea6-c66a05f844d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:19:59.276936 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 793/1617] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:19:59 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:19:59.280858 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8893033d-643b-4e55-8ea6-c66a05f844d9 None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:19:59.282020 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 825/1618] 10.222.0.55 () {60 vars in 1153 bytes} [Fri Sep 23 15:19:59 2022] POST /identity/v3/projects => generated 373 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:19:59.287100 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-8893033d-643b-4e55-8ea6-c66a05f844d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:19:59.294159 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f49d045-7bd5-4117-8b18-e92bdfea09d5 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:19:59.305514 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8893033d-643b-4e55-8ea6-c66a05f844d9 None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:19:59.379689 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edcbc13-1a7f-4d26-bb6d-6ad7a6324df8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd259178570864ffeb936ee10dfec6b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:19:59.380844 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edcbc13-1a7f-4d26-bb6d-6ad7a6324df8 service neutron] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:59.381155 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edcbc13-1a7f-4d26-bb6d-6ad7a6324df8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:59.381525 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edcbc13-1a7f-4d26-bb6d-6ad7a6324df8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:59.399401 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:19:59.400586 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f49d045-7bd5-4117-8b18-e92bdfea09d5 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:19:59.404408 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f948b486-ddce-4477-8fb4-cb1b6177885d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd259178570864ffeb936ee10dfec6b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:19:59.409494 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f948b486-ddce-4477-8fb4-cb1b6177885d service neutron] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:59.409494 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f948b486-ddce-4477-8fb4-cb1b6177885d service neutron] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:59.409494 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f948b486-ddce-4477-8fb4-cb1b6177885d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:59.441085 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:19:59.459640 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edcbc13-1a7f-4d26-bb6d-6ad7a6324df8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:19:59.461171 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edcbc13-1a7f-4d26-bb6d-6ad7a6324df8 service neutron] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:19:59.472720 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 794/1619] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:19:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:19:59.473631 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edcbc13-1a7f-4d26-bb6d-6ad7a6324df8 service neutron] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:19:59.474576 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f948b486-ddce-4477-8fb4-cb1b6177885d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:19:59.476289 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f948b486-ddce-4477-8fb4-cb1b6177885d service neutron] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:19:59.486660 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edcbc13-1a7f-4d26-bb6d-6ad7a6324df8 service neutron] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:19:59.490044 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 826/1620] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:19:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:19:59.491581 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f4fe2d-092b-4438-bbe4-11ab4f7514d0 None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:59.492071 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f4fe2d-092b-4438-bbe4-11ab4f7514d0 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:59.492513 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f4fe2d-092b-4438-bbe4-11ab4f7514d0 None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:59.493209 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04dade52-0c12-4dc9-ba6d-6f60c3a9ba5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:19:59.494076 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dade52-0c12-4dc9-ba6d-6f60c3a9ba5e admin admin] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:59.494434 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dade52-0c12-4dc9-ba6d-6f60c3a9ba5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:59.494801 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dade52-0c12-4dc9-ba6d-6f60c3a9ba5e admin admin] PATH_INFO: `/v3/users` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:59.495504 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04dade52-0c12-4dc9-ba6d-6f60c3a9ba5e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:19:59.496643 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 827/1621] 10.222.0.55 () {54 vars in 961 bytes} [Fri Sep 23 15:19:59 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:19:59.498093 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04dade52-0c12-4dc9-ba6d-6f60c3a9ba5e admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:19:59.498280 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea33c09-14cf-42fd-82ec-2594694161d8 None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:59.499807 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea33c09-14cf-42fd-82ec-2594694161d8 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:59.500310 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea33c09-14cf-42fd-82ec-2594694161d8 None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:59.502196 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 828/1622] 10.222.0.55 () {54 vars in 912 bytes} [Fri Sep 23 15:19:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:19:59.504030 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52284d55-e13a-41ce-8257-517669139527 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:59.506841 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52284d55-e13a-41ce-8257-517669139527 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:59.506841 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52284d55-e13a-41ce-8257-517669139527 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:59.530883 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-52284d55-e13a-41ce-8257-517669139527 None None] MFA Rules not processed for user `b3c6c7a1325a47be924268e1aa1a7a18`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:19:59.532234 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-52284d55-e13a-41ce-8257-517669139527 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:19:59.602833 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 829/1623] 10.222.0.55 () {58 vars in 1058 bytes} [Fri Sep 23 15:19:59 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:19:59.604042 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9b3446-02dc-4185-b484-1d6cb841b784 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:59.604334 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9b3446-02dc-4185-b484-1d6cb841b784 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:59.604640 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9b3446-02dc-4185-b484-1d6cb841b784 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:59.619006 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 795/1624] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Sep 23 15:19:59 2022] POST /identity/v3/users => generated 293 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:19:59.619006 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04dade52-0c12-4dc9-ba6d-6f60c3a9ba5e admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:19:59.635090 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04dade52-0c12-4dc9-ba6d-6f60c3a9ba5e admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:19:59.637847 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-7b9b3446-02dc-4185-b484-1d6cb841b784 None None] MFA Rules not processed for user `b3c6c7a1325a47be924268e1aa1a7a18`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:19:59.640710 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-7b9b3446-02dc-4185-b484-1d6cb841b784 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:19:59.643506 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed964ab-61e9-4e58-9449-5f0c3262d9dc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3c6c7a1325a47be924268e1aa1a7a18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:19:59.644429 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed964ab-61e9-4e58-9449-5f0c3262d9dc service nova] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:59.644847 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed964ab-61e9-4e58-9449-5f0c3262d9dc service nova] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:59.645296 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed964ab-61e9-4e58-9449-5f0c3262d9dc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:59.656802 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 830/1625] 10.222.0.55 () {58 vars in 1009 bytes} [Fri Sep 23 15:19:59 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:19:59.657678 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9b3446-02dc-4185-b484-1d6cb841b784 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:19:59.674710 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9b3446-02dc-4185-b484-1d6cb841b784 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:19:59.684068 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8119555-2f25-41e1-b39e-93669e92d896 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:19:59.685184 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8119555-2f25-41e1-b39e-93669e92d896 admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:59.685707 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8119555-2f25-41e1-b39e-93669e92d896 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:59.686787 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8119555-2f25-41e1-b39e-93669e92d896 admin admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:59.688776 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8119555-2f25-41e1-b39e-93669e92d896 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:19:59.690843 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8119555-2f25-41e1-b39e-93669e92d896 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:19:59.706889 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 831/1626] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Sep 23 15:19:59 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:19:59.709137 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8119555-2f25-41e1-b39e-93669e92d896 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:19:59.752614 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed964ab-61e9-4e58-9449-5f0c3262d9dc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:19:59.754860 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed964ab-61e9-4e58-9449-5f0c3262d9dc service nova] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:19:59.769125 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 796/1627] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:19:59 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:19:59.769841 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed964ab-61e9-4e58-9449-5f0c3262d9dc service nova] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:19:59.790359 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed964ab-61e9-4e58-9449-5f0c3262d9dc service nova] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:19:59.803886 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe6d503-555b-45c6-8654-3b7def2dbc50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:19:59.805665 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe6d503-555b-45c6-8654-3b7def2dbc50 admin admin] REQUEST_METHOD: `PUT` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:59.805955 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe6d503-555b-45c6-8654-3b7def2dbc50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:59.809965 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe6d503-555b-45c6-8654-3b7def2dbc50 admin admin] PATH_INFO: `/v3/projects/d6b3cf777856438cad979cb178473936/users/551f2d4cd3994fd8a44f771d7cfb3cf2/roles/8e2919e79df040e8926f21607818b58c` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:59.847599 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8119555-2f25-41e1-b39e-93669e92d896 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:19:59.849955 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe6d503-555b-45c6-8654-3b7def2dbc50 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d6b3cf777856438cad979cb178473936, user_id=551f2d4cd3994fd8a44f771d7cfb3cf2, role_id=8e2919e79df040e8926f21607818b58c)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:19:59.859656 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c75206-7a9c-471d-b8b9-0f0378d0f0e2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feca6278be904ef8b0dc580a67e477f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:19:59.859960 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe6d503-555b-45c6-8654-3b7def2dbc50 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:19:59.861280 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c75206-7a9c-471d-b8b9-0f0378d0f0e2 service cinder] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:59.861566 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c75206-7a9c-471d-b8b9-0f0378d0f0e2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:59.861844 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c75206-7a9c-471d-b8b9-0f0378d0f0e2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:59.883449 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c75206-7a9c-471d-b8b9-0f0378d0f0e2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:19:59.884916 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c75206-7a9c-471d-b8b9-0f0378d0f0e2 service cinder] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:19:59.896992 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 832/1628] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:19:59 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:20:00.090089 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 797/1629] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Sep 23 15:19:59 2022] PUT /identity/v3/projects/d6b3cf777856438cad979cb178473936/users/551f2d4cd3994fd8a44f771d7cfb3cf2/roles/8e2919e79df040e8926f21607818b58c => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:20:00.126999 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c75206-7a9c-471d-b8b9-0f0378d0f0e2 service cinder] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:00.139228 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c75206-7a9c-471d-b8b9-0f0378d0f0e2 service cinder] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:00.207286 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1b3469-5482-4a8b-a9d4-ac473e282a00 None None] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:00.666172 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1b3469-5482-4a8b-a9d4-ac473e282a00 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:00.666172 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1b3469-5482-4a8b-a9d4-ac473e282a00 None None] PATH_INFO: `/` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:00.666172 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 798/1630] 10.222.0.55 () {54 vars in 912 bytes} [Fri Sep 23 15:20:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 23 15:20:00.666172 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3834b4bb-2811-42e1-bbca-48f7c6df0654 None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:00.666172 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3834b4bb-2811-42e1-bbca-48f7c6df0654 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:00.666172 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3834b4bb-2811-42e1-bbca-48f7c6df0654 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:00.672112 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-3834b4bb-2811-42e1-bbca-48f7c6df0654 None None] MFA Rules not processed for user `b3c6c7a1325a47be924268e1aa1a7a18`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:20:00.672112 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-3834b4bb-2811-42e1-bbca-48f7c6df0654 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:20:00.677709 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cc3d3a-5293-4d94-8ffa-3b61c31a050c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:00.677709 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cc3d3a-5293-4d94-8ffa-3b61c31a050c admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:00.677709 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cc3d3a-5293-4d94-8ffa-3b61c31a050c admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:00.677709 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cc3d3a-5293-4d94-8ffa-3b61c31a050c admin admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:00.678653 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cc3d3a-5293-4d94-8ffa-3b61c31a050c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:00.678653 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cc3d3a-5293-4d94-8ffa-3b61c31a050c admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:00.678653 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 833/1631] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Sep 23 15:20:00 2022] GET /identity/v3/roles => generated 1420 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:20:00.678653 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cc3d3a-5293-4d94-8ffa-3b61c31a050c admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:00.678653 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cc3d3a-5293-4d94-8ffa-3b61c31a050c admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:00.681022 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e679a56-eb0e-47e6-b2b7-f9a652a28cab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:00.681022 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e679a56-eb0e-47e6-b2b7-f9a652a28cab admin admin] REQUEST_METHOD: `PUT` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:00.681022 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e679a56-eb0e-47e6-b2b7-f9a652a28cab admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:00.681022 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e679a56-eb0e-47e6-b2b7-f9a652a28cab admin admin] PATH_INFO: `/v3/projects/d6b3cf777856438cad979cb178473936/users/551f2d4cd3994fd8a44f771d7cfb3cf2/roles/8e2919e79df040e8926f21607818b58c` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:00.682147 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e679a56-eb0e-47e6-b2b7-f9a652a28cab admin admin] RBAC: Authorizing `identity:create_grant(project_id=d6b3cf777856438cad979cb178473936, user_id=551f2d4cd3994fd8a44f771d7cfb3cf2, role_id=8e2919e79df040e8926f21607818b58c)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:00.682147 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e679a56-eb0e-47e6-b2b7-f9a652a28cab admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:00.682147 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 834/1632] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Sep 23 15:20:00 2022] PUT /identity/v3/projects/d6b3cf777856438cad979cb178473936/users/551f2d4cd3994fd8a44f771d7cfb3cf2/roles/8e2919e79df040e8926f21607818b58c => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:20:00.682147 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a68f75-ea53-4a0b-8020-6dc39a29bc3c None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:00.682147 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a68f75-ea53-4a0b-8020-6dc39a29bc3c None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:00.682147 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a68f75-ea53-4a0b-8020-6dc39a29bc3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:00.683275 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-55a68f75-ea53-4a0b-8020-6dc39a29bc3c None None] MFA Rules not processed for user `551f2d4cd3994fd8a44f771d7cfb3cf2`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:20:00.683275 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-55a68f75-ea53-4a0b-8020-6dc39a29bc3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:20:00.683770 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 799/1633] 10.222.0.55 () {58 vars in 1009 bytes} [Fri Sep 23 15:20:00 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 15:20:00.683770 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3834b4bb-2811-42e1-bbca-48f7c6df0654 None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:00.683770 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3834b4bb-2811-42e1-bbca-48f7c6df0654 None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:00.684647 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 835/1634] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:20:00 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:20:00.685679 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2654d90-00fc-4d3a-bc8a-6a3ea61d17ab service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feca6278be904ef8b0dc580a67e477f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:00.685679 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2654d90-00fc-4d3a-bc8a-6a3ea61d17ab service cinder] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:00.685679 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2654d90-00fc-4d3a-bc8a-6a3ea61d17ab service cinder] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:00.685679 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2654d90-00fc-4d3a-bc8a-6a3ea61d17ab service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:00.687669 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2654d90-00fc-4d3a-bc8a-6a3ea61d17ab service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:00.687669 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2654d90-00fc-4d3a-bc8a-6a3ea61d17ab service cinder] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:00.702373 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 800/1635] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:20:00 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 264 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:20:00.772179 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:20:00.787163 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a68f75-ea53-4a0b-8020-6dc39a29bc3c None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:00.803343 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a68f75-ea53-4a0b-8020-6dc39a29bc3c None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:00.812732 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a994d7-1bc8-4a2d-aa1d-5a542a69ab0b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feca6278be904ef8b0dc580a67e477f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:00.814827 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a994d7-1bc8-4a2d-aa1d-5a542a69ab0b service cinder] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:00.816658 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a994d7-1bc8-4a2d-aa1d-5a542a69ab0b service cinder] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:00.817380 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a994d7-1bc8-4a2d-aa1d-5a542a69ab0b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:00.836282 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a994d7-1bc8-4a2d-aa1d-5a542a69ab0b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:00.838130 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a994d7-1bc8-4a2d-aa1d-5a542a69ab0b service cinder] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:00.863000 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 836/1636] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:20:00 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:20:01.355631 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2654d90-00fc-4d3a-bc8a-6a3ea61d17ab service cinder] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:01.364872 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a994d7-1bc8-4a2d-aa1d-5a542a69ab0b service cinder] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:01.374940 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2654d90-00fc-4d3a-bc8a-6a3ea61d17ab service cinder] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:01.381393 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7a6814-3025-4cef-9985-618d862185b9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feca6278be904ef8b0dc580a67e477f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:01.382141 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7a6814-3025-4cef-9985-618d862185b9 service cinder] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:01.382530 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7a6814-3025-4cef-9985-618d862185b9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:01.382909 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7a6814-3025-4cef-9985-618d862185b9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:01.490431 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a994d7-1bc8-4a2d-aa1d-5a542a69ab0b service cinder] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:01.502440 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bc42fe-81f2-471c-ad0e-42a5d147a6da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:01.503753 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bc42fe-81f2-471c-ad0e-42a5d147a6da admin admin] REQUEST_METHOD: `DELETE` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:01.504307 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bc42fe-81f2-471c-ad0e-42a5d147a6da admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:01.504698 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bc42fe-81f2-471c-ad0e-42a5d147a6da admin admin] PATH_INFO: `/v3/users/473a152c83444c87aea51c0c1569b249` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:01.506823 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bc42fe-81f2-471c-ad0e-42a5d147a6da admin admin] RBAC: Authorizing `identity:delete_user(user_id=473a152c83444c87aea51c0c1569b249)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:01.510300 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bc42fe-81f2-471c-ad0e-42a5d147a6da admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:01.587372 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7a6814-3025-4cef-9985-618d862185b9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:01.593195 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7a6814-3025-4cef-9985-618d862185b9 service cinder] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:01.606116 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 801/1637] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:20:01 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 255 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:20:01.804547 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-b0bc42fe-81f2-471c-ad0e-42a5d147a6da admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '473a152c83444c87aea51c0c1569b249'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:01.806565 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:20:01.843831 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-b0bc42fe-81f2-471c-ad0e-42a5d147a6da admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '473a152c83444c87aea51c0c1569b249'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:01.855007 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-b0bc42fe-81f2-471c-ad0e-42a5d147a6da admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '473a152c83444c87aea51c0c1569b249'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:02.022720 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-b0bc42fe-81f2-471c-ad0e-42a5d147a6da admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '473a152c83444c87aea51c0c1569b249'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:02.024716 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-b0bc42fe-81f2-471c-ad0e-42a5d147a6da admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '473a152c83444c87aea51c0c1569b249'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:02.040058 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 837/1638] 10.222.0.55 () {58 vars in 1224 bytes} [Fri Sep 23 15:20:01 2022] DELETE /identity/v3/users/473a152c83444c87aea51c0c1569b249 => generated 0 bytes in 675 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 15:20:02.046605 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baacd3f8-33ca-43b1-a335-71d0d6828612 None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:02.047066 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baacd3f8-33ca-43b1-a335-71d0d6828612 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:02.048173 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baacd3f8-33ca-43b1-a335-71d0d6828612 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:02.079757 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-baacd3f8-33ca-43b1-a335-71d0d6828612 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:20:02.079757 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-baacd3f8-33ca-43b1-a335-71d0d6828612 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:20:02.155408 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 802/1639] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:20:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 15:20:02.169152 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bc42fe-81f2-471c-ad0e-42a5d147a6da admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:02.171177 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-b0bc42fe-81f2-471c-ad0e-42a5d147a6da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:20:02.244818 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bc42fe-81f2-471c-ad0e-42a5d147a6da admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:02.248594 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45124525-3b00-45ba-9cb1-c2c28c8ef040 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd259178570864ffeb936ee10dfec6b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:02.249221 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45124525-3b00-45ba-9cb1-c2c28c8ef040 service neutron] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:02.249507 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45124525-3b00-45ba-9cb1-c2c28c8ef040 service neutron] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:02.249777 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45124525-3b00-45ba-9cb1-c2c28c8ef040 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:02.259459 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45124525-3b00-45ba-9cb1-c2c28c8ef040 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:02.261168 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45124525-3b00-45ba-9cb1-c2c28c8ef040 service neutron] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:02.272605 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 838/1640] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:20:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:20:02.339329 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:20:02.531547 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a771b245-7f39-44e0-a05b-40981f5d3a25 None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:02.531923 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a771b245-7f39-44e0-a05b-40981f5d3a25 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:02.532521 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a771b245-7f39-44e0-a05b-40981f5d3a25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:02.572243 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:20:02.578259 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-a771b245-7f39-44e0-a05b-40981f5d3a25 None None] MFA Rules not processed for user `5c802d31a1304331bf82f4904956dbc4`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:20:02.579763 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-a771b245-7f39-44e0-a05b-40981f5d3a25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:20:02.744514 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45124525-3b00-45ba-9cb1-c2c28c8ef040 service neutron] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:02.747291 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-45124525-3b00-45ba-9cb1-c2c28c8ef040 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:20:02.751162 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 803/1641] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:20:02 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:20:02.766198 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45124525-3b00-45ba-9cb1-c2c28c8ef040 service neutron] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:02.772816 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf3c221-7780-498b-82fc-f062afa35aac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:02.773634 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf3c221-7780-498b-82fc-f062afa35aac admin admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:02.774013 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf3c221-7780-498b-82fc-f062afa35aac admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:02.774411 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf3c221-7780-498b-82fc-f062afa35aac admin admin] PATH_INFO: `/v3/projects` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:02.775406 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf3c221-7780-498b-82fc-f062afa35aac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:02.778232 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf3c221-7780-498b-82fc-f062afa35aac admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:03.003468 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 839/1642] 10.222.0.55 () {60 vars in 1153 bytes} [Fri Sep 23 15:20:02 2022] POST /identity/v3/projects => generated 403 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:20:03.004494 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a771b245-7f39-44e0-a05b-40981f5d3a25 None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:03.017989 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a771b245-7f39-44e0-a05b-40981f5d3a25 None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:03.095227 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf9a876-7c44-4695-82e1-dda50fd91e75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:03.095999 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf9a876-7c44-4695-82e1-dda50fd91e75 admin admin] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:03.096950 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf9a876-7c44-4695-82e1-dda50fd91e75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:03.096950 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf9a876-7c44-4695-82e1-dda50fd91e75 admin admin] PATH_INFO: `/v3/users` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:03.097213 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf9a876-7c44-4695-82e1-dda50fd91e75 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:03.100105 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf9a876-7c44-4695-82e1-dda50fd91e75 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:03.235850 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 804/1643] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Sep 23 15:20:03 2022] POST /identity/v3/users => generated 308 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 23 15:20:03.237234 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf3c221-7780-498b-82fc-f062afa35aac admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:03.247843 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf3c221-7780-498b-82fc-f062afa35aac admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:03.253798 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44689a12-c46c-4e51-a0ed-7b37be994bb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:03.254438 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44689a12-c46c-4e51-a0ed-7b37be994bb5 admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:03.254912 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44689a12-c46c-4e51-a0ed-7b37be994bb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:03.255191 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44689a12-c46c-4e51-a0ed-7b37be994bb5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:03.255839 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44689a12-c46c-4e51-a0ed-7b37be994bb5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:03.257564 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44689a12-c46c-4e51-a0ed-7b37be994bb5 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:03.274521 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 840/1644] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Sep 23 15:20:03 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:20:03.275763 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf9a876-7c44-4695-82e1-dda50fd91e75 admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:03.286881 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf9a876-7c44-4695-82e1-dda50fd91e75 admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:03.291509 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d817e6-c9f3-4bcd-8db6-096e029411e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:03.292210 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d817e6-c9f3-4bcd-8db6-096e029411e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:03.292493 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d817e6-c9f3-4bcd-8db6-096e029411e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:03.292770 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d817e6-c9f3-4bcd-8db6-096e029411e1 admin admin] PATH_INFO: `/v3/projects/6d87e738bf9b4194aba97c37ad6ee55d/users/279b144747134e21bffa06553daef280/roles/8e2919e79df040e8926f21607818b58c` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:03.304542 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:20:03.314337 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d817e6-c9f3-4bcd-8db6-096e029411e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d87e738bf9b4194aba97c37ad6ee55d, user_id=279b144747134e21bffa06553daef280, role_id=8e2919e79df040e8926f21607818b58c)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:03.316894 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d817e6-c9f3-4bcd-8db6-096e029411e1 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:03.419659 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 805/1645] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Sep 23 15:20:03 2022] PUT /identity/v3/projects/6d87e738bf9b4194aba97c37ad6ee55d/users/279b144747134e21bffa06553daef280/roles/8e2919e79df040e8926f21607818b58c => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:20:03.422246 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44689a12-c46c-4e51-a0ed-7b37be994bb5 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:03.432461 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44689a12-c46c-4e51-a0ed-7b37be994bb5 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:03.501617 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac513ef-fc6e-4f1e-9593-049c19971e06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:03.502255 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac513ef-fc6e-4f1e-9593-049c19971e06 admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:03.502546 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac513ef-fc6e-4f1e-9593-049c19971e06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:03.502814 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac513ef-fc6e-4f1e-9593-049c19971e06 admin admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:03.503463 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac513ef-fc6e-4f1e-9593-049c19971e06 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:03.505274 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac513ef-fc6e-4f1e-9593-049c19971e06 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:03.520054 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 841/1646] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Sep 23 15:20:03 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:20:03.522633 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d817e6-c9f3-4bcd-8db6-096e029411e1 admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:03.541721 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d817e6-c9f3-4bcd-8db6-096e029411e1 admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:03.557268 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dafc042-5d1a-4f25-ac67-c13612438bc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:03.561731 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dafc042-5d1a-4f25-ac67-c13612438bc2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:03.561731 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dafc042-5d1a-4f25-ac67-c13612438bc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:03.561731 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dafc042-5d1a-4f25-ac67-c13612438bc2 admin admin] PATH_INFO: `/v3/projects/6d87e738bf9b4194aba97c37ad6ee55d/users/279b144747134e21bffa06553daef280/roles/8e2919e79df040e8926f21607818b58c` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:03.563499 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dafc042-5d1a-4f25-ac67-c13612438bc2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d87e738bf9b4194aba97c37ad6ee55d, user_id=279b144747134e21bffa06553daef280, role_id=8e2919e79df040e8926f21607818b58c)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:03.573024 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dafc042-5d1a-4f25-ac67-c13612438bc2 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:03.591774 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 806/1647] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Sep 23 15:20:03 2022] PUT /identity/v3/projects/6d87e738bf9b4194aba97c37ad6ee55d/users/279b144747134e21bffa06553daef280/roles/8e2919e79df040e8926f21607818b58c => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:20:03.911032 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac513ef-fc6e-4f1e-9593-049c19971e06 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:03.914088 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-9ac513ef-fc6e-4f1e-9593-049c19971e06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:20:03.957252 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac513ef-fc6e-4f1e-9593-049c19971e06 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:04.111782 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fe757a-a4de-4c4c-a9c1-14e5eef17de2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:04.114250 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fe757a-a4de-4c4c-a9c1-14e5eef17de2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:04.114250 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fe757a-a4de-4c4c-a9c1-14e5eef17de2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:04.114250 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fe757a-a4de-4c4c-a9c1-14e5eef17de2 admin admin] PATH_INFO: `/v3/projects/700da9a5171446dfb55095c96caa714f` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:04.114869 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fe757a-a4de-4c4c-a9c1-14e5eef17de2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=700da9a5171446dfb55095c96caa714f)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:04.116353 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: WARNING py.warnings [None req-47fe757a-a4de-4c4c-a9c1-14e5eef17de2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 23 15:20:04.116353 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: warnings.warn(msg) Sep 23 15:20:04.116353 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: Sep 23 15:20:04.117691 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fe757a-a4de-4c4c-a9c1-14e5eef17de2 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:04.461628 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-47fe757a-a4de-4c4c-a9c1-14e5eef17de2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '700da9a5171446dfb55095c96caa714f', 'request_id': 'req-47fe757a-a4de-4c4c-a9c1-14e5eef17de2'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:04.547405 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-47fe757a-a4de-4c4c-a9c1-14e5eef17de2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '700da9a5171446dfb55095c96caa714f', 'request_id': 'req-47fe757a-a4de-4c4c-a9c1-14e5eef17de2'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:04.588928 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-47fe757a-a4de-4c4c-a9c1-14e5eef17de2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '700da9a5171446dfb55095c96caa714f', 'request_id': 'req-47fe757a-a4de-4c4c-a9c1-14e5eef17de2'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:04.793532 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-47fe757a-a4de-4c4c-a9c1-14e5eef17de2 admin admin] The token cache is being invalidate because project 700da9a5171446dfb55095c96caa714f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81994) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 15:20:04.796915 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-47fe757a-a4de-4c4c-a9c1-14e5eef17de2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:04.798400 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-47fe757a-a4de-4c4c-a9c1-14e5eef17de2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:04.809349 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 842/1648] 10.222.0.55 () {58 vars in 1233 bytes} [Fri Sep 23 15:20:03 2022] DELETE /identity/v3/projects/700da9a5171446dfb55095c96caa714f => generated 0 bytes in 899 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:20:04.814160 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b709e588-76c9-4be2-8dde-54c534f30ef1 None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:04.814575 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b709e588-76c9-4be2-8dde-54c534f30ef1 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:04.814781 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b709e588-76c9-4be2-8dde-54c534f30ef1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:04.855692 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-b709e588-76c9-4be2-8dde-54c534f30ef1 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:20:04.857396 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-b709e588-76c9-4be2-8dde-54c534f30ef1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:20:04.984787 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 807/1649] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:20:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:20:04.988572 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c361ef43-6c64-483f-9f46-ad297c6d6af5 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:04.989136 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c361ef43-6c64-483f-9f46-ad297c6d6af5 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:04.989607 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c361ef43-6c64-483f-9f46-ad297c6d6af5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:05.021954 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-c361ef43-6c64-483f-9f46-ad297c6d6af5 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:20:05.024817 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-c361ef43-6c64-483f-9f46-ad297c6d6af5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:20:05.036434 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 843/1650] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:20:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:20:05.040380 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b709e588-76c9-4be2-8dde-54c534f30ef1 None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:05.052380 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b709e588-76c9-4be2-8dde-54c534f30ef1 None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:05.057625 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb594be3-0a9b-45fd-89ef-b9f5d50979f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:05.059065 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb594be3-0a9b-45fd-89ef-b9f5d50979f5 admin admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:05.059381 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb594be3-0a9b-45fd-89ef-b9f5d50979f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:05.059658 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb594be3-0a9b-45fd-89ef-b9f5d50979f5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:05.060364 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb594be3-0a9b-45fd-89ef-b9f5d50979f5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:20:05.061300 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb594be3-0a9b-45fd-89ef-b9f5d50979f5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:05.066705 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb594be3-0a9b-45fd-89ef-b9f5d50979f5 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:05.185385 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 808/1651] 10.222.0.55 () {58 vars in 1153 bytes} [Fri Sep 23 15:20:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:20:05.193647 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c361ef43-6c64-483f-9f46-ad297c6d6af5 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:05.211553 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c361ef43-6c64-483f-9f46-ad297c6d6af5 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:05.219872 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc65bd8-d27d-4239-b8dd-30a528c46f0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:05.259866 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc65bd8-d27d-4239-b8dd-30a528c46f0b admin admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:05.259866 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc65bd8-d27d-4239-b8dd-30a528c46f0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:05.259866 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc65bd8-d27d-4239-b8dd-30a528c46f0b admin admin] PATH_INFO: `/v3/projects` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:05.259866 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc65bd8-d27d-4239-b8dd-30a528c46f0b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:05.259866 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc65bd8-d27d-4239-b8dd-30a528c46f0b admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:05.451581 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 844/1652] 10.222.0.55 () {60 vars in 1153 bytes} [Fri Sep 23 15:20:05 2022] POST /identity/v3/projects => generated 369 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:20:05.456566 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb594be3-0a9b-45fd-89ef-b9f5d50979f5 admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:05.466696 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb594be3-0a9b-45fd-89ef-b9f5d50979f5 admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:05.575284 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04f6f57-4d91-4e43-a379-7ce22841bda0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:05.578126 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04f6f57-4d91-4e43-a379-7ce22841bda0 admin admin] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:05.579097 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04f6f57-4d91-4e43-a379-7ce22841bda0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:05.579652 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04f6f57-4d91-4e43-a379-7ce22841bda0 admin admin] PATH_INFO: `/v3/users` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:05.582639 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04f6f57-4d91-4e43-a379-7ce22841bda0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:05.587509 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04f6f57-4d91-4e43-a379-7ce22841bda0 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:05.729086 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 809/1653] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Sep 23 15:20:05 2022] POST /identity/v3/users => generated 285 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:20:05.730350 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc65bd8-d27d-4239-b8dd-30a528c46f0b admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:05.748190 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc65bd8-d27d-4239-b8dd-30a528c46f0b admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:05.762856 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0c1550-ba79-4fde-8af0-39728dbc1b7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:05.764215 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0c1550-ba79-4fde-8af0-39728dbc1b7e admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:05.765736 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0c1550-ba79-4fde-8af0-39728dbc1b7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:05.766704 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0c1550-ba79-4fde-8af0-39728dbc1b7e admin admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:05.769937 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0c1550-ba79-4fde-8af0-39728dbc1b7e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:05.772427 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0c1550-ba79-4fde-8af0-39728dbc1b7e admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:05.802664 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 845/1654] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Sep 23 15:20:05 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:20:05.806343 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04f6f57-4d91-4e43-a379-7ce22841bda0 admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:05.852536 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04f6f57-4d91-4e43-a379-7ce22841bda0 admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:05.866290 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54b507f-b914-48c7-ac04-e1ac5960fd18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:05.866974 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54b507f-b914-48c7-ac04-e1ac5960fd18 admin admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:05.867309 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54b507f-b914-48c7-ac04-e1ac5960fd18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:05.867589 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54b507f-b914-48c7-ac04-e1ac5960fd18 admin admin] PATH_INFO: `/v3/roles` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:05.868324 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54b507f-b914-48c7-ac04-e1ac5960fd18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:05.870226 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54b507f-b914-48c7-ac04-e1ac5960fd18 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:05.889584 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 810/1655] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Sep 23 15:20:05 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 23 15:20:05.894004 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0c1550-ba79-4fde-8af0-39728dbc1b7e admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:05.908699 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0c1550-ba79-4fde-8af0-39728dbc1b7e admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:05.923559 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b11fd6f-3d23-4f15-a607-ec73a4c909a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:05.928923 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b11fd6f-3d23-4f15-a607-ec73a4c909a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:05.929490 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b11fd6f-3d23-4f15-a607-ec73a4c909a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:05.929784 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b11fd6f-3d23-4f15-a607-ec73a4c909a0 admin admin] PATH_INFO: `/v3/projects/78cb5c6b72204875bd53bd69fb691391/users/f20a767247014581bfab4cf08efd1d07/roles/af257be2dee54d6b942f58650196582c` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:05.959132 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b11fd6f-3d23-4f15-a607-ec73a4c909a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=78cb5c6b72204875bd53bd69fb691391, user_id=f20a767247014581bfab4cf08efd1d07, role_id=af257be2dee54d6b942f58650196582c)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:05.962297 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b11fd6f-3d23-4f15-a607-ec73a4c909a0 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:05.969868 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:DELETE=5,keystone:SELECT=93,keystone:INSERT=2 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:20:06.118404 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 846/1656] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Sep 23 15:20:05 2022] PUT /identity/v3/projects/78cb5c6b72204875bd53bd69fb691391/users/f20a767247014581bfab4cf08efd1d07/roles/af257be2dee54d6b942f58650196582c => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 15:20:06.132837 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1824f346-c6af-49ec-a7ec-16dd43958a62 None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:06.133124 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1824f346-c6af-49ec-a7ec-16dd43958a62 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:06.133406 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1824f346-c6af-49ec-a7ec-16dd43958a62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:06.236130 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-1824f346-c6af-49ec-a7ec-16dd43958a62 None None] MFA Rules not processed for user `f20a767247014581bfab4cf08efd1d07`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:20:06.237726 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-1824f346-c6af-49ec-a7ec-16dd43958a62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:20:06.322603 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:20:06.410639 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 811/1657] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:20:06 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 278 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:20:06.421007 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b11fd6f-3d23-4f15-a607-ec73a4c909a0 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:06.561693 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b11fd6f-3d23-4f15-a607-ec73a4c909a0 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:06.577040 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57e28af-1b52-4f66-9a34-a601275fcbc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:06.577040 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57e28af-1b52-4f66-9a34-a601275fcbc4 admin admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:06.577040 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57e28af-1b52-4f66-9a34-a601275fcbc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:06.577040 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57e28af-1b52-4f66-9a34-a601275fcbc4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:06.577484 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57e28af-1b52-4f66-9a34-a601275fcbc4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:06.577484 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57e28af-1b52-4f66-9a34-a601275fcbc4 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:06.741140 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 847/1658] 10.222.0.55 () {60 vars in 1153 bytes} [Fri Sep 23 15:20:06 2022] POST /identity/v3/projects => generated 369 bytes in 321 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:20:06.742131 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1824f346-c6af-49ec-a7ec-16dd43958a62 None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:06.761282 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1824f346-c6af-49ec-a7ec-16dd43958a62 None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:06.856067 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6238a53c-66df-4dc2-baf7-bf61b175e046 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:06.856986 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6238a53c-66df-4dc2-baf7-bf61b175e046 admin admin] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:06.857279 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6238a53c-66df-4dc2-baf7-bf61b175e046 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:06.857603 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6238a53c-66df-4dc2-baf7-bf61b175e046 admin admin] PATH_INFO: `/v3/users` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:06.858252 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6238a53c-66df-4dc2-baf7-bf61b175e046 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:06.860233 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6238a53c-66df-4dc2-baf7-bf61b175e046 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:07.039254 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 812/1659] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Sep 23 15:20:06 2022] POST /identity/v3/users => generated 291 bytes in 297 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:20:07.042407 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57e28af-1b52-4f66-9a34-a601275fcbc4 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:07.065162 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57e28af-1b52-4f66-9a34-a601275fcbc4 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:07.072211 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f1a765-7160-4fae-a235-4f4051873b6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:07.073067 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f1a765-7160-4fae-a235-4f4051873b6c admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:07.073681 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f1a765-7160-4fae-a235-4f4051873b6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:07.074671 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f1a765-7160-4fae-a235-4f4051873b6c admin admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:07.075927 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f1a765-7160-4fae-a235-4f4051873b6c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:07.081508 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f1a765-7160-4fae-a235-4f4051873b6c admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:07.096633 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 848/1660] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Sep 23 15:20:07 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:20:07.100974 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6238a53c-66df-4dc2-baf7-bf61b175e046 admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:07.116746 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6238a53c-66df-4dc2-baf7-bf61b175e046 admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:07.132058 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23c8470-ed50-4f01-9bd5-5595d612746d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:07.139584 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23c8470-ed50-4f01-9bd5-5595d612746d admin admin] REQUEST_METHOD: `PUT` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:07.140425 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23c8470-ed50-4f01-9bd5-5595d612746d admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:07.140425 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23c8470-ed50-4f01-9bd5-5595d612746d admin admin] PATH_INFO: `/v3/projects/66bc72c8c228479cb1df423a2586d00f/users/24a77813e4874d7ab67491832b1eaf5b/roles/8e2919e79df040e8926f21607818b58c` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:07.196932 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23c8470-ed50-4f01-9bd5-5595d612746d admin admin] RBAC: Authorizing `identity:create_grant(project_id=66bc72c8c228479cb1df423a2586d00f, user_id=24a77813e4874d7ab67491832b1eaf5b, role_id=8e2919e79df040e8926f21607818b58c)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:07.199286 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23c8470-ed50-4f01-9bd5-5595d612746d admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:07.305746 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 813/1661] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Sep 23 15:20:07 2022] PUT /identity/v3/projects/66bc72c8c228479cb1df423a2586d00f/users/24a77813e4874d7ab67491832b1eaf5b/roles/8e2919e79df040e8926f21607818b58c => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:20:07.307484 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f1a765-7160-4fae-a235-4f4051873b6c admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:07.320626 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f1a765-7160-4fae-a235-4f4051873b6c admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:07.402187 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5674b4cd-ef24-4fff-8d5d-2263077518e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:07.403118 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5674b4cd-ef24-4fff-8d5d-2263077518e5 admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:07.403597 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5674b4cd-ef24-4fff-8d5d-2263077518e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:07.404067 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5674b4cd-ef24-4fff-8d5d-2263077518e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:07.405091 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5674b4cd-ef24-4fff-8d5d-2263077518e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:07.406896 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5674b4cd-ef24-4fff-8d5d-2263077518e5 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:07.428838 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23c8470-ed50-4f01-9bd5-5595d612746d admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:07.429269 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 849/1662] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Sep 23 15:20:07 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:20:07.447615 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23c8470-ed50-4f01-9bd5-5595d612746d admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:07.463654 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d174c18-94c9-4f05-843f-da6395f64e30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:07.465035 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d174c18-94c9-4f05-843f-da6395f64e30 admin admin] REQUEST_METHOD: `PUT` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:07.465035 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d174c18-94c9-4f05-843f-da6395f64e30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:07.465306 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d174c18-94c9-4f05-843f-da6395f64e30 admin admin] PATH_INFO: `/v3/projects/66bc72c8c228479cb1df423a2586d00f/users/24a77813e4874d7ab67491832b1eaf5b/roles/8e2919e79df040e8926f21607818b58c` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:07.476221 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d174c18-94c9-4f05-843f-da6395f64e30 admin admin] RBAC: Authorizing `identity:create_grant(project_id=66bc72c8c228479cb1df423a2586d00f, user_id=24a77813e4874d7ab67491832b1eaf5b, role_id=8e2919e79df040e8926f21607818b58c)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:07.479538 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d174c18-94c9-4f05-843f-da6395f64e30 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:07.508552 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 814/1663] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Sep 23 15:20:07 2022] PUT /identity/v3/projects/66bc72c8c228479cb1df423a2586d00f/users/24a77813e4874d7ab67491832b1eaf5b/roles/8e2919e79df040e8926f21607818b58c => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:20:07.522046 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdc91dd-dd8a-4fc6-bafb-2430b4d75999 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:07.522341 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdc91dd-dd8a-4fc6-bafb-2430b4d75999 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:07.522618 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdc91dd-dd8a-4fc6-bafb-2430b4d75999 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:07.573481 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-4fdc91dd-dd8a-4fc6-bafb-2430b4d75999 None None] MFA Rules not processed for user `24a77813e4874d7ab67491832b1eaf5b`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:20:07.574720 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-4fdc91dd-dd8a-4fc6-bafb-2430b4d75999 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:20:07.666221 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8efdf3-0cee-46f0-93de-7d79f8d8bc14 None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:07.666494 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8efdf3-0cee-46f0-93de-7d79f8d8bc14 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:07.666808 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8efdf3-0cee-46f0-93de-7d79f8d8bc14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:07.739523 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-2e8efdf3-0cee-46f0-93de-7d79f8d8bc14 None None] MFA Rules not processed for user `279b144747134e21bffa06553daef280`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:20:07.742278 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-2e8efdf3-0cee-46f0-93de-7d79f8d8bc14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:20:07.777535 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 850/1664] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:20:07 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:20:07.789345 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdc91dd-dd8a-4fc6-bafb-2430b4d75999 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:07.791044 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-4fdc91dd-dd8a-4fc6-bafb-2430b4d75999 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:20:07.962400 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdc91dd-dd8a-4fc6-bafb-2430b4d75999 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:07.984421 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5ec24d-db3a-4bbb-974c-dbb07429a692 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feca6278be904ef8b0dc580a67e477f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:07.985135 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5ec24d-db3a-4bbb-974c-dbb07429a692 service cinder] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:07.985407 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5ec24d-db3a-4bbb-974c-dbb07429a692 service cinder] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:07.985683 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5ec24d-db3a-4bbb-974c-dbb07429a692 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:08.005303 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 815/1665] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:20:07 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 340 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 15:20:08.026656 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5ec24d-db3a-4bbb-974c-dbb07429a692 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:08.028159 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5ec24d-db3a-4bbb-974c-dbb07429a692 service cinder] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:08.065881 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 851/1666] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:20:07 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 277 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 23 15:20:08.603448 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:20:09.531661 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43bdc54-7ecf-417b-ab2b-e9e67095d6e5 None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:09.540080 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43bdc54-7ecf-417b-ab2b-e9e67095d6e5 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:09.540080 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43bdc54-7ecf-417b-ab2b-e9e67095d6e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:09.569091 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:20:09.612544 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-f43bdc54-7ecf-417b-ab2b-e9e67095d6e5 None None] MFA Rules not processed for user `f5f0ecaf38e74c96bc439fde75d93d37`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:20:09.616837 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-f43bdc54-7ecf-417b-ab2b-e9e67095d6e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:20:09.738033 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 816/1667] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:20:09 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:20:09.757565 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5ec24d-db3a-4bbb-974c-dbb07429a692 service cinder] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:09.760586 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-eb5ec24d-db3a-4bbb-974c-dbb07429a692 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:20:09.980198 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5ec24d-db3a-4bbb-974c-dbb07429a692 service cinder] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:09.986985 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893f41de-132c-44a5-b331-5e6207d62004 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3c6c7a1325a47be924268e1aa1a7a18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:09.987424 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893f41de-132c-44a5-b331-5e6207d62004 service nova] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:09.988047 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893f41de-132c-44a5-b331-5e6207d62004 service nova] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:09.988156 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893f41de-132c-44a5-b331-5e6207d62004 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:10.009082 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893f41de-132c-44a5-b331-5e6207d62004 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:10.017578 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893f41de-132c-44a5-b331-5e6207d62004 service nova] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:10.039675 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 852/1668] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:20:09 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 276 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:20:11.068233 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18d3d3c-9d16-4936-9765-72f1bd49cd97 None None] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:11.068816 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18d3d3c-9d16-4936-9765-72f1bd49cd97 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:11.068816 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18d3d3c-9d16-4936-9765-72f1bd49cd97 None None] PATH_INFO: `/` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:11.070111 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 817/1669] 10.222.0.55 () {54 vars in 921 bytes} [Fri Sep 23 15:20:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:20:11.092415 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf71e533-7804-45c9-89be-28990493ee96 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:11.093010 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf71e533-7804-45c9-89be-28990493ee96 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:11.093010 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf71e533-7804-45c9-89be-28990493ee96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:11.121039 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-bf71e533-7804-45c9-89be-28990493ee96 None None] MFA Rules not processed for user `09f1a55d180642c781a6e0b017d9f0aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:20:11.123722 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-bf71e533-7804-45c9-89be-28990493ee96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:20:11.254118 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 853/1670] 10.222.0.55 () {58 vars in 1018 bytes} [Fri Sep 23 15:20:11 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:20:11.271248 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18d3d3c-9d16-4936-9765-72f1bd49cd97 None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:11.273366 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-d18d3d3c-9d16-4936-9765-72f1bd49cd97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:20:11.309178 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18d3d3c-9d16-4936-9765-72f1bd49cd97 None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:11.325651 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fe22da-1531-44d3-8a38-dbc9226c2573 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09f1a55d180642c781a6e0b017d9f0aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:11.325651 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fe22da-1531-44d3-8a38-dbc9226c2573 service placement] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:11.325651 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fe22da-1531-44d3-8a38-dbc9226c2573 service placement] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:11.325651 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fe22da-1531-44d3-8a38-dbc9226c2573 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:11.352726 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fe22da-1531-44d3-8a38-dbc9226c2573 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:11.354929 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fe22da-1531-44d3-8a38-dbc9226c2573 service placement] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:11.374762 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 818/1671] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:20:11 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:20:11.940546 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:20:12.971628 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ec1dc6-216b-4820-81e5-faf6747da3cf None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:12.972637 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ec1dc6-216b-4820-81e5-faf6747da3cf None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:12.973059 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ec1dc6-216b-4820-81e5-faf6747da3cf None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:12.974809 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 854/1672] 10.222.0.55 () {54 vars in 921 bytes} [Fri Sep 23 15:20:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:20:13.104746 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f98cb5-10db-4323-8fbf-d7181ace8a47 None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:13.104746 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f98cb5-10db-4323-8fbf-d7181ace8a47 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:13.105379 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f98cb5-10db-4323-8fbf-d7181ace8a47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:13.156258 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-e9f98cb5-10db-4323-8fbf-d7181ace8a47 None None] MFA Rules not processed for user `09f1a55d180642c781a6e0b017d9f0aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:20:13.157728 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-e9f98cb5-10db-4323-8fbf-d7181ace8a47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:20:13.174706 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 819/1673] 10.222.0.55 () {58 vars in 1018 bytes} [Fri Sep 23 15:20:13 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:20:13.182577 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ec1dc6-216b-4820-81e5-faf6747da3cf None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:13.213894 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ec1dc6-216b-4820-81e5-faf6747da3cf None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:13.224283 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5086457c-1b23-4b86-85d9-f56c554796d2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09f1a55d180642c781a6e0b017d9f0aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:13.225020 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5086457c-1b23-4b86-85d9-f56c554796d2 service placement] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:13.225384 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5086457c-1b23-4b86-85d9-f56c554796d2 service placement] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:13.225746 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5086457c-1b23-4b86-85d9-f56c554796d2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:13.244914 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5086457c-1b23-4b86-85d9-f56c554796d2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:13.246543 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5086457c-1b23-4b86-85d9-f56c554796d2 service placement] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:13.317600 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f98cb5-10db-4323-8fbf-d7181ace8a47 None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:13.319338 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-e9f98cb5-10db-4323-8fbf-d7181ace8a47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:20:13.346059 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 855/1674] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:20:13 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:20:13.531466 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f98cb5-10db-4323-8fbf-d7181ace8a47 None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:13.546332 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae662b9-7b91-4db8-ad80-0bee1ec662af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:13.547344 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae662b9-7b91-4db8-ad80-0bee1ec662af admin admin] REQUEST_METHOD: `DELETE` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:13.547721 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae662b9-7b91-4db8-ad80-0bee1ec662af admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:13.548233 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae662b9-7b91-4db8-ad80-0bee1ec662af admin admin] PATH_INFO: `/v3/users/5c802d31a1304331bf82f4904956dbc4` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:13.554754 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae662b9-7b91-4db8-ad80-0bee1ec662af admin admin] RBAC: Authorizing `identity:delete_user(user_id=5c802d31a1304331bf82f4904956dbc4)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:13.556025 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: WARNING py.warnings [None req-1ae662b9-7b91-4db8-ad80-0bee1ec662af admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 23 15:20:13.556025 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: warnings.warn(msg) Sep 23 15:20:13.556025 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: Sep 23 15:20:13.557531 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae662b9-7b91-4db8-ad80-0bee1ec662af admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:13.869144 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5086457c-1b23-4b86-85d9-f56c554796d2 service placement] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:13.880399 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5086457c-1b23-4b86-85d9-f56c554796d2 service placement] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:13.895885 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995ad4d5-51c5-4f00-8f21-2e77b0bef305 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:13.897261 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995ad4d5-51c5-4f00-8f21-2e77b0bef305 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:13.897750 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995ad4d5-51c5-4f00-8f21-2e77b0bef305 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:13.898603 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995ad4d5-51c5-4f00-8f21-2e77b0bef305 admin admin] PATH_INFO: `/v3/users/f20a767247014581bfab4cf08efd1d07` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:13.900579 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995ad4d5-51c5-4f00-8f21-2e77b0bef305 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f20a767247014581bfab4cf08efd1d07)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:13.903180 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995ad4d5-51c5-4f00-8f21-2e77b0bef305 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:13.961659 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-1ae662b9-7b91-4db8-ad80-0bee1ec662af admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5c802d31a1304331bf82f4904956dbc4'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:14.028492 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-1ae662b9-7b91-4db8-ad80-0bee1ec662af admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5c802d31a1304331bf82f4904956dbc4'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:14.054981 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-1ae662b9-7b91-4db8-ad80-0bee1ec662af admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5c802d31a1304331bf82f4904956dbc4'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:14.060189 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:20:14.204778 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-1ae662b9-7b91-4db8-ad80-0bee1ec662af admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5c802d31a1304331bf82f4904956dbc4'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:14.205569 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-1ae662b9-7b91-4db8-ad80-0bee1ec662af admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5c802d31a1304331bf82f4904956dbc4'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:14.221925 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 820/1675] 10.222.0.55 () {58 vars in 1224 bytes} [Fri Sep 23 15:20:13 2022] DELETE /identity/v3/users/5c802d31a1304331bf82f4904956dbc4 => generated 0 bytes in 905 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 15:20:14.222806 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae662b9-7b91-4db8-ad80-0bee1ec662af admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:14.229300 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-1ae662b9-7b91-4db8-ad80-0bee1ec662af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:20:14.238373 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-995ad4d5-51c5-4f00-8f21-2e77b0bef305 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f20a767247014581bfab4cf08efd1d07'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:14.244656 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:20:14.268693 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-995ad4d5-51c5-4f00-8f21-2e77b0bef305 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f20a767247014581bfab4cf08efd1d07'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:14.286003 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-995ad4d5-51c5-4f00-8f21-2e77b0bef305 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f20a767247014581bfab4cf08efd1d07'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:14.418886 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae662b9-7b91-4db8-ad80-0bee1ec662af admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:14.422505 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-995ad4d5-51c5-4f00-8f21-2e77b0bef305 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f20a767247014581bfab4cf08efd1d07'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:14.423339 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-995ad4d5-51c5-4f00-8f21-2e77b0bef305 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f20a767247014581bfab4cf08efd1d07'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:14.435726 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 856/1676] 10.222.0.55 () {58 vars in 1224 bytes} [Fri Sep 23 15:20:13 2022] DELETE /identity/v3/users/f20a767247014581bfab4cf08efd1d07 => generated 0 bytes in 563 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 15:20:14.436103 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6892e78d-9bc0-484b-90bb-448cb50bdc95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:14.436103 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6892e78d-9bc0-484b-90bb-448cb50bdc95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:14.436103 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6892e78d-9bc0-484b-90bb-448cb50bdc95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:14.436103 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6892e78d-9bc0-484b-90bb-448cb50bdc95 admin admin] PATH_INFO: `/v3/users/c500d775ea144aa48ba2b4db12ec3407` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:14.437661 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb76f49-8cd2-476a-893b-4deb540623cc None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:14.438087 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb76f49-8cd2-476a-893b-4deb540623cc None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:14.438671 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb76f49-8cd2-476a-893b-4deb540623cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:14.438790 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6892e78d-9bc0-484b-90bb-448cb50bdc95 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c500d775ea144aa48ba2b4db12ec3407)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:14.440790 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6892e78d-9bc0-484b-90bb-448cb50bdc95 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:14.478367 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-aeb76f49-8cd2-476a-893b-4deb540623cc None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:20:14.480100 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-aeb76f49-8cd2-476a-893b-4deb540623cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:20:14.611130 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 857/1677] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:20:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 15:20:14.643374 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb76f49-8cd2-476a-893b-4deb540623cc None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:14.644771 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-aeb76f49-8cd2-476a-893b-4deb540623cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:20:14.751168 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb76f49-8cd2-476a-893b-4deb540623cc None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:14.758504 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37ce10e-deee-45a8-b07a-1b304f3d0447 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd259178570864ffeb936ee10dfec6b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:14.782354 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37ce10e-deee-45a8-b07a-1b304f3d0447 service neutron] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:14.782354 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37ce10e-deee-45a8-b07a-1b304f3d0447 service neutron] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:14.782354 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37ce10e-deee-45a8-b07a-1b304f3d0447 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:14.786462 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37ce10e-deee-45a8-b07a-1b304f3d0447 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:14.787892 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37ce10e-deee-45a8-b07a-1b304f3d0447 service neutron] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:14.799194 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 858/1678] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:20:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:20:14.963953 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-6892e78d-9bc0-484b-90bb-448cb50bdc95 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c500d775ea144aa48ba2b4db12ec3407'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:14.993530 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-6892e78d-9bc0-484b-90bb-448cb50bdc95 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c500d775ea144aa48ba2b4db12ec3407'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:15.007496 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37ce10e-deee-45a8-b07a-1b304f3d0447 service neutron] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:15.009113 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-d37ce10e-deee-45a8-b07a-1b304f3d0447 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:20:15.015761 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-6892e78d-9bc0-484b-90bb-448cb50bdc95 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c500d775ea144aa48ba2b4db12ec3407'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:15.037730 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37ce10e-deee-45a8-b07a-1b304f3d0447 service neutron] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:15.054415 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cb1cee-b442-407c-a701-efbac909fdc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:15.054765 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cb1cee-b442-407c-a701-efbac909fdc6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:15.055052 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cb1cee-b442-407c-a701-efbac909fdc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:15.055369 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cb1cee-b442-407c-a701-efbac909fdc6 admin admin] PATH_INFO: `/v3/projects/14696ebba19a41fcbd4d5fcda48bec1e` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:15.059408 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1cb1cee-b442-407c-a701-efbac909fdc6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=14696ebba19a41fcbd4d5fcda48bec1e)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:15.063182 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1cb1cee-b442-407c-a701-efbac909fdc6 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:15.158156 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-6892e78d-9bc0-484b-90bb-448cb50bdc95 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c500d775ea144aa48ba2b4db12ec3407'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:15.160384 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-6892e78d-9bc0-484b-90bb-448cb50bdc95 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c500d775ea144aa48ba2b4db12ec3407'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:15.179948 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 821/1679] 10.222.0.55 () {58 vars in 1224 bytes} [Fri Sep 23 15:20:14 2022] DELETE /identity/v3/users/c500d775ea144aa48ba2b4db12ec3407 => generated 0 bytes in 957 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:20:15.187107 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01ca470-a810-45f1-818e-0b4355731236 None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:15.187396 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01ca470-a810-45f1-818e-0b4355731236 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:15.187673 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01ca470-a810-45f1-818e-0b4355731236 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:15.260070 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-f01ca470-a810-45f1-818e-0b4355731236 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:20:15.264349 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-c1cb1cee-b442-407c-a701-efbac909fdc6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '14696ebba19a41fcbd4d5fcda48bec1e', 'request_id': 'req-c1cb1cee-b442-407c-a701-efbac909fdc6'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:15.265878 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-f01ca470-a810-45f1-818e-0b4355731236 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:20:15.294584 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-c1cb1cee-b442-407c-a701-efbac909fdc6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '14696ebba19a41fcbd4d5fcda48bec1e', 'request_id': 'req-c1cb1cee-b442-407c-a701-efbac909fdc6'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:15.318281 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-c1cb1cee-b442-407c-a701-efbac909fdc6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '14696ebba19a41fcbd4d5fcda48bec1e', 'request_id': 'req-c1cb1cee-b442-407c-a701-efbac909fdc6'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:15.423700 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-c1cb1cee-b442-407c-a701-efbac909fdc6 admin admin] The token cache is being invalidate because project 14696ebba19a41fcbd4d5fcda48bec1e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81994) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 15:20:15.424501 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-c1cb1cee-b442-407c-a701-efbac909fdc6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:15.428745 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-c1cb1cee-b442-407c-a701-efbac909fdc6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:15.436047 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 859/1680] 10.222.0.55 () {58 vars in 1233 bytes} [Fri Sep 23 15:20:15 2022] DELETE /identity/v3/projects/14696ebba19a41fcbd4d5fcda48bec1e => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:20:15.437203 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cb1cee-b442-407c-a701-efbac909fdc6 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:15.442462 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-c1cb1cee-b442-407c-a701-efbac909fdc6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:20:15.451765 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 822/1681] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:20:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:20:15.499389 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cb1cee-b442-407c-a701-efbac909fdc6 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:15.514199 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01ca470-a810-45f1-818e-0b4355731236 None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:15.517176 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-f01ca470-a810-45f1-818e-0b4355731236 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:20:15.532547 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953f7adf-76af-4a15-ac12-a14560e41b81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:15.533555 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953f7adf-76af-4a15-ac12-a14560e41b81 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:15.534023 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953f7adf-76af-4a15-ac12-a14560e41b81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:15.534472 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953f7adf-76af-4a15-ac12-a14560e41b81 admin admin] PATH_INFO: `/v3/users/279b144747134e21bffa06553daef280` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:15.536129 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953f7adf-76af-4a15-ac12-a14560e41b81 admin admin] RBAC: Authorizing `identity:delete_user(user_id=279b144747134e21bffa06553daef280)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:15.538279 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953f7adf-76af-4a15-ac12-a14560e41b81 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:15.671467 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01ca470-a810-45f1-818e-0b4355731236 None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:15.697669 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0c97c7-0aa9-4a6d-9319-3c4dbd26b5ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd259178570864ffeb936ee10dfec6b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:15.699062 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0c97c7-0aa9-4a6d-9319-3c4dbd26b5ca service neutron] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:15.699570 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0c97c7-0aa9-4a6d-9319-3c4dbd26b5ca service neutron] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:15.700159 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0c97c7-0aa9-4a6d-9319-3c4dbd26b5ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:15.717863 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0c97c7-0aa9-4a6d-9319-3c4dbd26b5ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:15.719867 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0c97c7-0aa9-4a6d-9319-3c4dbd26b5ca service neutron] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:15.750791 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 823/1682] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:20:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:20:16.023512 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-953f7adf-76af-4a15-ac12-a14560e41b81 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '279b144747134e21bffa06553daef280'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:16.081885 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-953f7adf-76af-4a15-ac12-a14560e41b81 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '279b144747134e21bffa06553daef280'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:16.132531 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-953f7adf-76af-4a15-ac12-a14560e41b81 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '279b144747134e21bffa06553daef280'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:16.252094 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-953f7adf-76af-4a15-ac12-a14560e41b81 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '279b144747134e21bffa06553daef280'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:16.256078 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-953f7adf-76af-4a15-ac12-a14560e41b81 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '279b144747134e21bffa06553daef280'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:16.273215 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 860/1683] 10.222.0.55 () {58 vars in 1224 bytes} [Fri Sep 23 15:20:15 2022] DELETE /identity/v3/users/279b144747134e21bffa06553daef280 => generated 0 bytes in 836 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:20:16.771321 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0c97c7-0aa9-4a6d-9319-3c4dbd26b5ca service neutron] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:16.773692 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-2b0c97c7-0aa9-4a6d-9319-3c4dbd26b5ca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:20:16.803082 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0c97c7-0aa9-4a6d-9319-3c4dbd26b5ca service neutron] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:16.980114 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca35ea33-9890-4b2a-8afe-59cc5719398c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:16.980907 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca35ea33-9890-4b2a-8afe-59cc5719398c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:16.981319 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca35ea33-9890-4b2a-8afe-59cc5719398c admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:16.982327 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca35ea33-9890-4b2a-8afe-59cc5719398c admin admin] PATH_INFO: `/v3/projects/78cb5c6b72204875bd53bd69fb691391` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:16.988312 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca35ea33-9890-4b2a-8afe-59cc5719398c admin admin] RBAC: Authorizing `identity:delete_project(project_id=78cb5c6b72204875bd53bd69fb691391)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:16.990865 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca35ea33-9890-4b2a-8afe-59cc5719398c admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:17.141116 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92ddfe0-3381-45c7-aad2-15bf186bab1c None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:17.141408 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92ddfe0-3381-45c7-aad2-15bf186bab1c None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:17.141959 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92ddfe0-3381-45c7-aad2-15bf186bab1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:17.216515 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-d92ddfe0-3381-45c7-aad2-15bf186bab1c None None] MFA Rules not processed for user `a5514abdc8b54282901367a4f2d02856`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:20:17.217997 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-d92ddfe0-3381-45c7-aad2-15bf186bab1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:20:17.331589 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-ca35ea33-9890-4b2a-8afe-59cc5719398c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '78cb5c6b72204875bd53bd69fb691391', 'request_id': 'req-ca35ea33-9890-4b2a-8afe-59cc5719398c'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:17.351181 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-ca35ea33-9890-4b2a-8afe-59cc5719398c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '78cb5c6b72204875bd53bd69fb691391', 'request_id': 'req-ca35ea33-9890-4b2a-8afe-59cc5719398c'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:17.397980 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 861/1684] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:20:17 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:20:17.402047 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2275a74f-d7d3-4a44-9a82-81fa39ed64ff None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:17.402530 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2275a74f-d7d3-4a44-9a82-81fa39ed64ff None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:17.404077 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2275a74f-d7d3-4a44-9a82-81fa39ed64ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:17.487754 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-2275a74f-d7d3-4a44-9a82-81fa39ed64ff None None] MFA Rules not processed for user `a5514abdc8b54282901367a4f2d02856`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:20:17.489565 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-2275a74f-d7d3-4a44-9a82-81fa39ed64ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:20:17.629555 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:20:17.655886 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 862/1685] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:20:17 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:20:17.667455 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2275a74f-d7d3-4a44-9a82-81fa39ed64ff None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:17.674977 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-2275a74f-d7d3-4a44-9a82-81fa39ed64ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:20:17.764730 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-ca35ea33-9890-4b2a-8afe-59cc5719398c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '78cb5c6b72204875bd53bd69fb691391', 'request_id': 'req-ca35ea33-9890-4b2a-8afe-59cc5719398c'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:17.779114 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-ca35ea33-9890-4b2a-8afe-59cc5719398c admin admin] The token cache is being invalidate because project 78cb5c6b72204875bd53bd69fb691391 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81993) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 15:20:17.781211 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-ca35ea33-9890-4b2a-8afe-59cc5719398c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:17.791960 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-ca35ea33-9890-4b2a-8afe-59cc5719398c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:17.796434 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 824/1686] 10.222.0.55 () {58 vars in 1233 bytes} [Fri Sep 23 15:20:16 2022] DELETE /identity/v3/projects/78cb5c6b72204875bd53bd69fb691391 => generated 0 bytes in 1023 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:20:17.798333 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca35ea33-9890-4b2a-8afe-59cc5719398c admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:17.800894 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-ca35ea33-9890-4b2a-8afe-59cc5719398c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:20:17.805295 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2275a74f-d7d3-4a44-9a82-81fa39ed64ff None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:17.812813 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36836d2-64fa-4439-8697-da8ac2e1b218 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3c6c7a1325a47be924268e1aa1a7a18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:17.813701 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36836d2-64fa-4439-8697-da8ac2e1b218 service nova] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:17.814144 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36836d2-64fa-4439-8697-da8ac2e1b218 service nova] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:17.814735 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36836d2-64fa-4439-8697-da8ac2e1b218 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:17.816885 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-b36836d2-64fa-4439-8697-da8ac2e1b218 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:20:17.830105 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36836d2-64fa-4439-8697-da8ac2e1b218 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:17.831711 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36836d2-64fa-4439-8697-da8ac2e1b218 service nova] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:17.844066 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 863/1687] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:20:17 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 23 15:20:17.845261 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36836d2-64fa-4439-8697-da8ac2e1b218 service nova] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:17.847007 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-b36836d2-64fa-4439-8697-da8ac2e1b218 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:20:18.060062 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca35ea33-9890-4b2a-8afe-59cc5719398c admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:18.065333 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9752c5-e396-4f24-bbbd-cec6d905f26a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:18.068313 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9752c5-e396-4f24-bbbd-cec6d905f26a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:18.068837 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9752c5-e396-4f24-bbbd-cec6d905f26a admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:18.071681 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9752c5-e396-4f24-bbbd-cec6d905f26a admin admin] PATH_INFO: `/v3/projects/6d87e738bf9b4194aba97c37ad6ee55d` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:18.080361 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9752c5-e396-4f24-bbbd-cec6d905f26a admin admin] RBAC: Authorizing `identity:delete_project(project_id=6d87e738bf9b4194aba97c37ad6ee55d)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:18.080361 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9752c5-e396-4f24-bbbd-cec6d905f26a admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:18.088400 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36836d2-64fa-4439-8697-da8ac2e1b218 service nova] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:18.095529 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50b4cdd-7b6f-44cd-acf2-c3877bd9649c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:18.096496 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50b4cdd-7b6f-44cd-acf2-c3877bd9649c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:18.096937 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50b4cdd-7b6f-44cd-acf2-c3877bd9649c admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:18.097395 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50b4cdd-7b6f-44cd-acf2-c3877bd9649c admin admin] PATH_INFO: `/v3/users/24a77813e4874d7ab67491832b1eaf5b` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:18.098950 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50b4cdd-7b6f-44cd-acf2-c3877bd9649c admin admin] RBAC: Authorizing `identity:delete_user(user_id=24a77813e4874d7ab67491832b1eaf5b)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:18.101479 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50b4cdd-7b6f-44cd-acf2-c3877bd9649c admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:18.386351 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-b50b4cdd-7b6f-44cd-acf2-c3877bd9649c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '24a77813e4874d7ab67491832b1eaf5b'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:18.387281 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-6d9752c5-e396-4f24-bbbd-cec6d905f26a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6d87e738bf9b4194aba97c37ad6ee55d', 'request_id': 'req-6d9752c5-e396-4f24-bbbd-cec6d905f26a'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:18.414385 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-6d9752c5-e396-4f24-bbbd-cec6d905f26a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6d87e738bf9b4194aba97c37ad6ee55d', 'request_id': 'req-6d9752c5-e396-4f24-bbbd-cec6d905f26a'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:18.425817 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-b50b4cdd-7b6f-44cd-acf2-c3877bd9649c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '24a77813e4874d7ab67491832b1eaf5b'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:18.475073 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-b50b4cdd-7b6f-44cd-acf2-c3877bd9649c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '24a77813e4874d7ab67491832b1eaf5b'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:18.553829 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-6d9752c5-e396-4f24-bbbd-cec6d905f26a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6d87e738bf9b4194aba97c37ad6ee55d', 'request_id': 'req-6d9752c5-e396-4f24-bbbd-cec6d905f26a'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:18.586206 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-6d9752c5-e396-4f24-bbbd-cec6d905f26a admin admin] The token cache is being invalidate because project 6d87e738bf9b4194aba97c37ad6ee55d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81993) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 15:20:18.586506 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-6d9752c5-e396-4f24-bbbd-cec6d905f26a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:18.587374 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-6d9752c5-e396-4f24-bbbd-cec6d905f26a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:18.589389 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 825/1688] 10.222.0.55 () {58 vars in 1233 bytes} [Fri Sep 23 15:20:17 2022] DELETE /identity/v3/projects/6d87e738bf9b4194aba97c37ad6ee55d => generated 0 bytes in 792 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:20:18.591311 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9752c5-e396-4f24-bbbd-cec6d905f26a admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:18.596843 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-6d9752c5-e396-4f24-bbbd-cec6d905f26a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:20:18.708642 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-b50b4cdd-7b6f-44cd-acf2-c3877bd9649c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '24a77813e4874d7ab67491832b1eaf5b'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:18.709469 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-b50b4cdd-7b6f-44cd-acf2-c3877bd9649c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '24a77813e4874d7ab67491832b1eaf5b'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:18.710574 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:20:18.720768 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 864/1689] 10.222.0.55 () {58 vars in 1224 bytes} [Fri Sep 23 15:20:17 2022] DELETE /identity/v3/users/24a77813e4874d7ab67491832b1eaf5b => generated 0 bytes in 875 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:20:18.722172 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5276ab29-cb7c-47a8-a5ab-a3ea55c85c9e None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:18.722463 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5276ab29-cb7c-47a8-a5ab-a3ea55c85c9e None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:18.723479 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5276ab29-cb7c-47a8-a5ab-a3ea55c85c9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:18.738385 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9752c5-e396-4f24-bbbd-cec6d905f26a admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:18.773166 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-5276ab29-cb7c-47a8-a5ab-a3ea55c85c9e None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:20:18.774414 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-5276ab29-cb7c-47a8-a5ab-a3ea55c85c9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:20:18.904705 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f7cbe3-30a7-4f8f-a72e-98b1b7afd1b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:18.905377 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f7cbe3-30a7-4f8f-a72e-98b1b7afd1b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:18.905664 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f7cbe3-30a7-4f8f-a72e-98b1b7afd1b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:18.905941 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f7cbe3-30a7-4f8f-a72e-98b1b7afd1b5 admin admin] PATH_INFO: `/v3/projects/f87732b5735542ec8efcd0f8d43a0e76` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:18.912750 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f7cbe3-30a7-4f8f-a72e-98b1b7afd1b5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f87732b5735542ec8efcd0f8d43a0e76)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:18.917623 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f7cbe3-30a7-4f8f-a72e-98b1b7afd1b5 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:18.968599 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 865/1690] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:20:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:20:18.970698 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b611741-172a-4b4e-99c7-69694efd3dc9 None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:18.971001 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b611741-172a-4b4e-99c7-69694efd3dc9 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:18.971284 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b611741-172a-4b4e-99c7-69694efd3dc9 None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:18.976383 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 866/1691] 10.222.0.55 () {54 vars in 912 bytes} [Fri Sep 23 15:20:18 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:20:18.977680 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4e70dd-99d7-4fc3-8987-2a9433949a61 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:18.979817 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4e70dd-99d7-4fc3-8987-2a9433949a61 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:18.980170 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4e70dd-99d7-4fc3-8987-2a9433949a61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:19.015208 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-1c4e70dd-99d7-4fc3-8987-2a9433949a61 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:20:19.016692 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-1c4e70dd-99d7-4fc3-8987-2a9433949a61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:20:19.037860 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 867/1692] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:20:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:20:19.039187 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c9d7fd-df85-4934-8471-911792c7af49 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:19.039481 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c9d7fd-df85-4934-8471-911792c7af49 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:19.039753 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c9d7fd-df85-4934-8471-911792c7af49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:19.116641 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-40c9d7fd-df85-4934-8471-911792c7af49 None None] MFA Rules not processed for user `d259178570864ffeb936ee10dfec6b28`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:20:19.120507 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-40c9d7fd-df85-4934-8471-911792c7af49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:20:19.123589 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-30f7cbe3-30a7-4f8f-a72e-98b1b7afd1b5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f87732b5735542ec8efcd0f8d43a0e76', 'request_id': 'req-30f7cbe3-30a7-4f8f-a72e-98b1b7afd1b5'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:19.142455 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-30f7cbe3-30a7-4f8f-a72e-98b1b7afd1b5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f87732b5735542ec8efcd0f8d43a0e76', 'request_id': 'req-30f7cbe3-30a7-4f8f-a72e-98b1b7afd1b5'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:19.208871 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 868/1693] 10.222.0.55 () {58 vars in 1009 bytes} [Fri Sep 23 15:20:19 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:20:19.210045 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c9d7fd-df85-4934-8471-911792c7af49 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:19.265374 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-30f7cbe3-30a7-4f8f-a72e-98b1b7afd1b5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f87732b5735542ec8efcd0f8d43a0e76', 'request_id': 'req-30f7cbe3-30a7-4f8f-a72e-98b1b7afd1b5'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:19.287538 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-30f7cbe3-30a7-4f8f-a72e-98b1b7afd1b5 admin admin] The token cache is being invalidate because project f87732b5735542ec8efcd0f8d43a0e76 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81993) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 15:20:19.287776 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-30f7cbe3-30a7-4f8f-a72e-98b1b7afd1b5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:19.290321 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-30f7cbe3-30a7-4f8f-a72e-98b1b7afd1b5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:19.291954 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 826/1694] 10.222.0.55 () {58 vars in 1233 bytes} [Fri Sep 23 15:20:18 2022] DELETE /identity/v3/projects/f87732b5735542ec8efcd0f8d43a0e76 => generated 0 bytes in 702 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:20:19.297568 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f7cbe3-30a7-4f8f-a72e-98b1b7afd1b5 admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:19.302605 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-30f7cbe3-30a7-4f8f-a72e-98b1b7afd1b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:20:19.333643 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f7cbe3-30a7-4f8f-a72e-98b1b7afd1b5 admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:19.343358 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c9d7fd-df85-4934-8471-911792c7af49 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:19.364945 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1baf54b7-e764-41f3-a53d-0619982d17cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd259178570864ffeb936ee10dfec6b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:19.366003 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1baf54b7-e764-41f3-a53d-0619982d17cf service neutron] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:19.366429 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1baf54b7-e764-41f3-a53d-0619982d17cf service neutron] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:19.366645 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1baf54b7-e764-41f3-a53d-0619982d17cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:19.373577 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-1baf54b7-e764-41f3-a53d-0619982d17cf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:20:19.377724 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadc1dd5-dd20-4837-8d30-c16fec4a24ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:19.380115 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadc1dd5-dd20-4837-8d30-c16fec4a24ce admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:19.380531 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadc1dd5-dd20-4837-8d30-c16fec4a24ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:19.380980 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadc1dd5-dd20-4837-8d30-c16fec4a24ce admin admin] PATH_INFO: `/v3/domains` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:19.383970 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadc1dd5-dd20-4837-8d30-c16fec4a24ce admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:20:19.384419 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadc1dd5-dd20-4837-8d30-c16fec4a24ce admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:19.397754 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadc1dd5-dd20-4837-8d30-c16fec4a24ce admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:19.405026 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1baf54b7-e764-41f3-a53d-0619982d17cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:19.410520 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1baf54b7-e764-41f3-a53d-0619982d17cf service neutron] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:19.427418 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 869/1695] 10.222.0.55 () {58 vars in 1153 bytes} [Fri Sep 23 15:20:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:20:19.429467 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4414d7b-cdc4-448e-97ea-e42c3d9cede9 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:19.429895 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4414d7b-cdc4-448e-97ea-e42c3d9cede9 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:19.430457 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4414d7b-cdc4-448e-97ea-e42c3d9cede9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:19.436618 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 827/1696] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:20:19 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:20:19.442057 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c939aaed-6b14-4acf-badf-a6915b79aa02 None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:19.443533 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c939aaed-6b14-4acf-badf-a6915b79aa02 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:19.443533 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c939aaed-6b14-4acf-badf-a6915b79aa02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:19.469305 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-f4414d7b-cdc4-448e-97ea-e42c3d9cede9 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:20:19.472331 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-f4414d7b-cdc4-448e-97ea-e42c3d9cede9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:20:19.512363 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 870/1697] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:20:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:20:19.513648 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4414d7b-cdc4-448e-97ea-e42c3d9cede9 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:19.517552 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-f4414d7b-cdc4-448e-97ea-e42c3d9cede9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:20:19.519542 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-c939aaed-6b14-4acf-badf-a6915b79aa02 None None] MFA Rules not processed for user `42dc378333ae4bbdbd42ce041ba15d3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:20:19.524321 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-c939aaed-6b14-4acf-badf-a6915b79aa02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:20:19.534357 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4414d7b-cdc4-448e-97ea-e42c3d9cede9 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:19.540153 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe1fb54-e40d-46da-9a6d-a0d06772838d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:19.541471 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe1fb54-e40d-46da-9a6d-a0d06772838d admin admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:19.541837 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe1fb54-e40d-46da-9a6d-a0d06772838d admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:19.542804 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe1fb54-e40d-46da-9a6d-a0d06772838d admin admin] PATH_INFO: `/v3/projects` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:19.543730 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe1fb54-e40d-46da-9a6d-a0d06772838d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:19.547728 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe1fb54-e40d-46da-9a6d-a0d06772838d admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:19.661826 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 828/1698] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:20:19 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:20:19.671472 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dd8a25-54e4-465b-8e22-5ed4fbe81a1c None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:19.671783 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dd8a25-54e4-465b-8e22-5ed4fbe81a1c None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:19.672115 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dd8a25-54e4-465b-8e22-5ed4fbe81a1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:19.705673 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-54dd8a25-54e4-465b-8e22-5ed4fbe81a1c None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:20:19.716350 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-54dd8a25-54e4-465b-8e22-5ed4fbe81a1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:20:19.716499 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 871/1699] 10.222.0.55 () {60 vars in 1153 bytes} [Fri Sep 23 15:20:19 2022] POST /identity/v3/projects => generated 365 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:20:19.721067 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe1fb54-e40d-46da-9a6d-a0d06772838d admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:19.726068 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-3fe1fb54-e40d-46da-9a6d-a0d06772838d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:20:19.729589 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 829/1700] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:20:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:20:19.730697 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54dd8a25-54e4-465b-8e22-5ed4fbe81a1c None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:19.734908 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-54dd8a25-54e4-465b-8e22-5ed4fbe81a1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:20:19.865581 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe1fb54-e40d-46da-9a6d-a0d06772838d admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:19.866135 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665f6384-16c9-42f4-aad1-ca5b9dc3625e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:19.868295 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665f6384-16c9-42f4-aad1-ca5b9dc3625e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:19.868922 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665f6384-16c9-42f4-aad1-ca5b9dc3625e admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:19.869442 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665f6384-16c9-42f4-aad1-ca5b9dc3625e admin admin] PATH_INFO: `/v3/projects/66bc72c8c228479cb1df423a2586d00f` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:19.873443 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665f6384-16c9-42f4-aad1-ca5b9dc3625e admin admin] RBAC: Authorizing `identity:delete_project(project_id=66bc72c8c228479cb1df423a2586d00f)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:19.881236 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665f6384-16c9-42f4-aad1-ca5b9dc3625e admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:19.936633 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54dd8a25-54e4-465b-8e22-5ed4fbe81a1c None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:20.103281 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbbb122-2460-4bc9-919d-a4f7368822ba service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3c6c7a1325a47be924268e1aa1a7a18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:20.103925 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbbb122-2460-4bc9-919d-a4f7368822ba service nova] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:20.104234 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbbb122-2460-4bc9-919d-a4f7368822ba service nova] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:20.104511 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbbb122-2460-4bc9-919d-a4f7368822ba service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:20.115264 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-665f6384-16c9-42f4-aad1-ca5b9dc3625e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '66bc72c8c228479cb1df423a2586d00f', 'request_id': 'req-665f6384-16c9-42f4-aad1-ca5b9dc3625e'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:20.128549 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-665f6384-16c9-42f4-aad1-ca5b9dc3625e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '66bc72c8c228479cb1df423a2586d00f', 'request_id': 'req-665f6384-16c9-42f4-aad1-ca5b9dc3625e'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:20.151004 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-665f6384-16c9-42f4-aad1-ca5b9dc3625e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '66bc72c8c228479cb1df423a2586d00f', 'request_id': 'req-665f6384-16c9-42f4-aad1-ca5b9dc3625e'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:20.215090 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbbb122-2460-4bc9-919d-a4f7368822ba service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:20.219542 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbbb122-2460-4bc9-919d-a4f7368822ba service nova] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:20.230575 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 830/1701] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:20:19 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 500 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:20:20.232112 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbbb122-2460-4bc9-919d-a4f7368822ba service nova] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:20.242444 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbbb122-2460-4bc9-919d-a4f7368822ba service nova] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:20.294802 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-665f6384-16c9-42f4-aad1-ca5b9dc3625e admin admin] The token cache is being invalidate because project 66bc72c8c228479cb1df423a2586d00f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81994) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 15:20:20.294802 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-665f6384-16c9-42f4-aad1-ca5b9dc3625e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:20.294802 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-665f6384-16c9-42f4-aad1-ca5b9dc3625e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:20.295883 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 872/1702] 10.222.0.55 () {58 vars in 1233 bytes} [Fri Sep 23 15:20:19 2022] DELETE /identity/v3/projects/66bc72c8c228479cb1df423a2586d00f => generated 0 bytes in 576 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:20:20.297589 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665f6384-16c9-42f4-aad1-ca5b9dc3625e admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:20.302052 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-665f6384-16c9-42f4-aad1-ca5b9dc3625e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:20:20.353646 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef19727-3762-4ce0-ade2-fbba91ad5ffa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:20.354426 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef19727-3762-4ce0-ade2-fbba91ad5ffa admin admin] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:20.354715 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef19727-3762-4ce0-ade2-fbba91ad5ffa admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:20.354993 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef19727-3762-4ce0-ade2-fbba91ad5ffa admin admin] PATH_INFO: `/v3/users` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:20.355690 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef19727-3762-4ce0-ade2-fbba91ad5ffa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:20.357847 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef19727-3762-4ce0-ade2-fbba91ad5ffa admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:20.450130 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665f6384-16c9-42f4-aad1-ca5b9dc3625e admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:20.459962 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc1f013-d22e-4844-8de2-b0b2b823923a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:20.460662 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc1f013-d22e-4844-8de2-b0b2b823923a admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:20.460943 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc1f013-d22e-4844-8de2-b0b2b823923a admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:20.461223 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc1f013-d22e-4844-8de2-b0b2b823923a admin admin] PATH_INFO: `/v3/domains` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:20.461884 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc1f013-d22e-4844-8de2-b0b2b823923a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:20:20.462192 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc1f013-d22e-4844-8de2-b0b2b823923a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:20.464222 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc1f013-d22e-4844-8de2-b0b2b823923a admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:20.482685 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 873/1703] 10.222.0.55 () {58 vars in 1153 bytes} [Fri Sep 23 15:20:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:20:20.482685 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0009658d-3087-4bf4-add5-1f6faad0c0ad None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:20.482685 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0009658d-3087-4bf4-add5-1f6faad0c0ad None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:20.482685 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0009658d-3087-4bf4-add5-1f6faad0c0ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:20.500986 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-0009658d-3087-4bf4-add5-1f6faad0c0ad None None] MFA Rules not processed for user `f5f0ecaf38e74c96bc439fde75d93d37`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:20:20.502211 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-0009658d-3087-4bf4-add5-1f6faad0c0ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:20:20.511639 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 831/1704] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Sep 23 15:20:20 2022] POST /identity/v3/users => generated 283 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:20:20.513636 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0629a979-6173-4499-b9bb-abcbbb53b06c None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:20.516650 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0629a979-6173-4499-b9bb-abcbbb53b06c None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:20.516650 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0629a979-6173-4499-b9bb-abcbbb53b06c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:20.597662 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-0629a979-6173-4499-b9bb-abcbbb53b06c None None] MFA Rules not processed for user `69dc5c52e36841f5aad028fc6a4b3daf`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:20:20.598185 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-0629a979-6173-4499-b9bb-abcbbb53b06c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:20:20.645963 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 874/1705] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:20:20 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:20:20.647379 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00c3d9b-4c6c-4cde-8bde-25ff33cf8b15 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:20.647675 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00c3d9b-4c6c-4cde-8bde-25ff33cf8b15 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:20.647954 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00c3d9b-4c6c-4cde-8bde-25ff33cf8b15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:20.673961 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-a00c3d9b-4c6c-4cde-8bde-25ff33cf8b15 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:20:20.675152 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-a00c3d9b-4c6c-4cde-8bde-25ff33cf8b15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:20:20.689890 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 875/1706] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:20:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:20:20.690895 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00c3d9b-4c6c-4cde-8bde-25ff33cf8b15 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:20.701053 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 832/1707] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:20:20 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:20:20.701569 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0629a979-6173-4499-b9bb-abcbbb53b06c None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:20.715218 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-a00c3d9b-4c6c-4cde-8bde-25ff33cf8b15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:20:20.727491 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0629a979-6173-4499-b9bb-abcbbb53b06c None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:20.770334 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8912dc23-a430-4990-a9cb-fc5a7f1f57f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:20.770334 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8912dc23-a430-4990-a9cb-fc5a7f1f57f3 admin admin] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:20.770334 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8912dc23-a430-4990-a9cb-fc5a7f1f57f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:20.770334 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8912dc23-a430-4990-a9cb-fc5a7f1f57f3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:20.771503 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8912dc23-a430-4990-a9cb-fc5a7f1f57f3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:20.771503 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8912dc23-a430-4990-a9cb-fc5a7f1f57f3 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:20.876424 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00c3d9b-4c6c-4cde-8bde-25ff33cf8b15 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:20.883195 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4300acd4-2784-4736-9097-6aec1547ab12 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09f1a55d180642c781a6e0b017d9f0aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:20.884772 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4300acd4-2784-4736-9097-6aec1547ab12 service placement] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:20.885099 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4300acd4-2784-4736-9097-6aec1547ab12 service placement] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:20.885380 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4300acd4-2784-4736-9097-6aec1547ab12 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:20.889803 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-4300acd4-2784-4736-9097-6aec1547ab12 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:20:20.910922 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4300acd4-2784-4736-9097-6aec1547ab12 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:20.912609 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4300acd4-2784-4736-9097-6aec1547ab12 service placement] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:20.927079 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 876/1708] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:20:20 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:20:20.928033 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4300acd4-2784-4736-9097-6aec1547ab12 service placement] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:20.929726 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-4300acd4-2784-4736-9097-6aec1547ab12 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:20:20.968257 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4300acd4-2784-4736-9097-6aec1547ab12 service placement] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:20.975554 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c547bfd-8236-4e69-bb7a-7d66fb57548b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:20.976748 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c547bfd-8236-4e69-bb7a-7d66fb57548b admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:20.977189 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c547bfd-8236-4e69-bb7a-7d66fb57548b admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:20.977625 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c547bfd-8236-4e69-bb7a-7d66fb57548b admin admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:20.978416 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c547bfd-8236-4e69-bb7a-7d66fb57548b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:20.980774 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c547bfd-8236-4e69-bb7a-7d66fb57548b admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:20.999544 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 877/1709] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Sep 23 15:20:20 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:20:21.010131 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c560a968-74bb-491d-91cf-68408c8fff8a None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:21.010429 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c560a968-74bb-491d-91cf-68408c8fff8a None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:21.011137 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c560a968-74bb-491d-91cf-68408c8fff8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:21.040517 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-c560a968-74bb-491d-91cf-68408c8fff8a None None] MFA Rules not processed for user `a5514abdc8b54282901367a4f2d02856`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:20:21.042003 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-c560a968-74bb-491d-91cf-68408c8fff8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:20:21.195037 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 833/1710] 10.222.0.55 () {60 vars in 1153 bytes} [Fri Sep 23 15:20:20 2022] POST /identity/v3/projects => generated 369 bytes in 492 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:20:21.195037 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624f03cf-eb1b-4b33-9d16-5a998a94a548 None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:21.195037 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624f03cf-eb1b-4b33-9d16-5a998a94a548 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:21.195037 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624f03cf-eb1b-4b33-9d16-5a998a94a548 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:21.253730 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-624f03cf-eb1b-4b33-9d16-5a998a94a548 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:20:21.258370 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-624f03cf-eb1b-4b33-9d16-5a998a94a548 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:20:21.265978 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 878/1711] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:20:20 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:20:21.273602 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8d7bae-51d6-4612-a135-5e7628ee5624 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:21.273907 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8d7bae-51d6-4612-a135-5e7628ee5624 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:21.275194 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8d7bae-51d6-4612-a135-5e7628ee5624 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:21.344309 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-2d8d7bae-51d6-4612-a135-5e7628ee5624 None None] MFA Rules not processed for user `69dc5c52e36841f5aad028fc6a4b3daf`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:20:21.348176 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-2d8d7bae-51d6-4612-a135-5e7628ee5624 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:20:21.430783 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 834/1712] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:20:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:20:21.434092 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624f03cf-eb1b-4b33-9d16-5a998a94a548 None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:21.462917 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624f03cf-eb1b-4b33-9d16-5a998a94a548 None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:21.478087 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5278dc0b-b070-4874-be0a-f3b64d9ea73d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:21.478700 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5278dc0b-b070-4874-be0a-f3b64d9ea73d admin admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:21.479374 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5278dc0b-b070-4874-be0a-f3b64d9ea73d admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:21.479374 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5278dc0b-b070-4874-be0a-f3b64d9ea73d admin admin] PATH_INFO: `/v3/roles` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:21.479930 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5278dc0b-b070-4874-be0a-f3b64d9ea73d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:21.481956 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5278dc0b-b070-4874-be0a-f3b64d9ea73d admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:21.507918 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 835/1713] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Sep 23 15:20:21 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:20:21.511414 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5278dc0b-b070-4874-be0a-f3b64d9ea73d admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:21.530054 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 879/1714] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:20:21 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:20:21.530746 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8d7bae-51d6-4612-a135-5e7628ee5624 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:21.539611 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5278dc0b-b070-4874-be0a-f3b64d9ea73d admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:21.557381 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8d7bae-51d6-4612-a135-5e7628ee5624 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:21.562456 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2dcb893-91c3-4e90-a3bd-03af7cadb3fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:21.563084 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dcb893-91c3-4e90-a3bd-03af7cadb3fb admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:21.563367 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dcb893-91c3-4e90-a3bd-03af7cadb3fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:21.563790 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dcb893-91c3-4e90-a3bd-03af7cadb3fb admin admin] PATH_INFO: `/v3/domains` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:21.564480 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2dcb893-91c3-4e90-a3bd-03af7cadb3fb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:20:21.570302 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2dcb893-91c3-4e90-a3bd-03af7cadb3fb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:21.570432 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903756ce-dfb5-4587-aa5e-4ea201278504 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:21.570432 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903756ce-dfb5-4587-aa5e-4ea201278504 admin admin] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:21.570432 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903756ce-dfb5-4587-aa5e-4ea201278504 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:21.570432 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903756ce-dfb5-4587-aa5e-4ea201278504 admin admin] PATH_INFO: `/v3/users` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:21.570725 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2dcb893-91c3-4e90-a3bd-03af7cadb3fb admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:21.570803 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903756ce-dfb5-4587-aa5e-4ea201278504 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:21.570803 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903756ce-dfb5-4587-aa5e-4ea201278504 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:21.601002 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 880/1715] 10.222.0.55 () {58 vars in 1153 bytes} [Fri Sep 23 15:20:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:20:21.601002 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dc924d-d4e9-44a1-add9-3c51ff713019 None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:21.601002 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dc924d-d4e9-44a1-add9-3c51ff713019 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:21.601002 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dc924d-d4e9-44a1-add9-3c51ff713019 None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:21.601002 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 881/1716] 10.222.0.55 () {54 vars in 912 bytes} [Fri Sep 23 15:20:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:20:21.601613 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dc924d-d4e9-44a1-add9-3c51ff713019 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:21.623130 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dc924d-d4e9-44a1-add9-3c51ff713019 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:21.630888 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7689a57-99cd-4b4f-b3e9-c9d05171c157 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:21.631567 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7689a57-99cd-4b4f-b3e9-c9d05171c157 admin admin] REQUEST_METHOD: `PUT` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:21.631851 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7689a57-99cd-4b4f-b3e9-c9d05171c157 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:21.632260 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7689a57-99cd-4b4f-b3e9-c9d05171c157 admin admin] PATH_INFO: `/v3/projects/70fee69f2c4b48bf80bf7b3667f52f59/users/1f8b374d0cc544559ae4458aedb5a412/roles/af257be2dee54d6b942f58650196582c` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:21.664404 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7689a57-99cd-4b4f-b3e9-c9d05171c157 admin admin] RBAC: Authorizing `identity:create_grant(project_id=70fee69f2c4b48bf80bf7b3667f52f59, user_id=1f8b374d0cc544559ae4458aedb5a412, role_id=af257be2dee54d6b942f58650196582c)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:21.667787 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7689a57-99cd-4b4f-b3e9-c9d05171c157 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:21.742521 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 836/1717] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Sep 23 15:20:21 2022] POST /identity/v3/users => generated 285 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:20:21.743631 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903756ce-dfb5-4587-aa5e-4ea201278504 admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:21.760222 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903756ce-dfb5-4587-aa5e-4ea201278504 admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:21.767262 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913e8129-0b21-42a9-83d5-2df999e706d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:21.768016 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913e8129-0b21-42a9-83d5-2df999e706d3 admin admin] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:21.768205 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913e8129-0b21-42a9-83d5-2df999e706d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:21.769120 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913e8129-0b21-42a9-83d5-2df999e706d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:21.769233 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913e8129-0b21-42a9-83d5-2df999e706d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:21.771135 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913e8129-0b21-42a9-83d5-2df999e706d3 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:21.789352 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 882/1718] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Sep 23 15:20:21 2022] PUT /identity/v3/projects/70fee69f2c4b48bf80bf7b3667f52f59/users/1f8b374d0cc544559ae4458aedb5a412/roles/af257be2dee54d6b942f58650196582c => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:20:21.791406 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1be7ca-2e04-476f-897f-6f56d25f49c6 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:21.791827 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1be7ca-2e04-476f-897f-6f56d25f49c6 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:21.792270 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1be7ca-2e04-476f-897f-6f56d25f49c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:21.824381 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-ab1be7ca-2e04-476f-897f-6f56d25f49c6 None None] MFA Rules not processed for user `d259178570864ffeb936ee10dfec6b28`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:20:21.825787 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-ab1be7ca-2e04-476f-897f-6f56d25f49c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:20:21.913124 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 837/1719] 10.222.0.55 () {60 vars in 1153 bytes} [Fri Sep 23 15:20:21 2022] POST /identity/v3/projects => generated 369 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:20:21.915509 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913e8129-0b21-42a9-83d5-2df999e706d3 admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:21.938382 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:20:21.948281 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 883/1720] 10.222.0.55 () {58 vars in 1009 bytes} [Fri Sep 23 15:20:21 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:20:21.953685 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2f68fc-4e1e-4f36-b9f3-6bd2c497464c None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:21.956806 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2f68fc-4e1e-4f36-b9f3-6bd2c497464c None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:21.958253 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2f68fc-4e1e-4f36-b9f3-6bd2c497464c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:22.054853 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-5a2f68fc-4e1e-4f36-b9f3-6bd2c497464c None None] MFA Rules not processed for user `1f8b374d0cc544559ae4458aedb5a412`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:20:22.066726 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-5a2f68fc-4e1e-4f36-b9f3-6bd2c497464c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:20:22.119966 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913e8129-0b21-42a9-83d5-2df999e706d3 admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:22.132784 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68222058-f51e-45af-b405-15efc1a0c2dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:22.133672 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68222058-f51e-45af-b405-15efc1a0c2dc admin admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:22.134025 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68222058-f51e-45af-b405-15efc1a0c2dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:22.134398 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68222058-f51e-45af-b405-15efc1a0c2dc admin admin] PATH_INFO: `/v3/roles` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:22.136794 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68222058-f51e-45af-b405-15efc1a0c2dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:22.141670 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68222058-f51e-45af-b405-15efc1a0c2dc admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:22.166695 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 838/1721] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Sep 23 15:20:21 2022] GET /identity/v3/roles => generated 1420 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:20:22.167776 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68222058-f51e-45af-b405-15efc1a0c2dc admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:22.196047 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68222058-f51e-45af-b405-15efc1a0c2dc admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:22.199756 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 884/1722] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:20:21 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:20:22.199756 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2f68fc-4e1e-4f36-b9f3-6bd2c497464c None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:22.200219 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-5a2f68fc-4e1e-4f36-b9f3-6bd2c497464c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:20:22.210640 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d196c7f-a386-4771-be78-e85f885b0947 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:22.211658 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d196c7f-a386-4771-be78-e85f885b0947 admin admin] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:22.212076 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d196c7f-a386-4771-be78-e85f885b0947 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:22.212256 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d196c7f-a386-4771-be78-e85f885b0947 admin admin] PATH_INFO: `/v3/users` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:22.212905 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d196c7f-a386-4771-be78-e85f885b0947 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:22.214894 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d196c7f-a386-4771-be78-e85f885b0947 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:22.371244 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2f68fc-4e1e-4f36-b9f3-6bd2c497464c None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:22.385252 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 839/1723] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Sep 23 15:20:22 2022] POST /identity/v3/users => generated 285 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:20:22.386050 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d196c7f-a386-4771-be78-e85f885b0947 admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:22.394610 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d1b5da-f177-460c-8937-8ed69849e2c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd259178570864ffeb936ee10dfec6b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:22.395267 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d1b5da-f177-460c-8937-8ed69849e2c9 service neutron] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:22.395548 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d1b5da-f177-460c-8937-8ed69849e2c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:22.395824 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d1b5da-f177-460c-8937-8ed69849e2c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:22.417710 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d196c7f-a386-4771-be78-e85f885b0947 admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:22.426406 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d1b5da-f177-460c-8937-8ed69849e2c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:22.427884 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d1b5da-f177-460c-8937-8ed69849e2c9 service neutron] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:22.439640 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c99d41-28ac-4b3f-85aa-305d36db7fb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:22.440635 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c99d41-28ac-4b3f-85aa-305d36db7fb7 admin admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:22.441001 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c99d41-28ac-4b3f-85aa-305d36db7fb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:22.441415 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c99d41-28ac-4b3f-85aa-305d36db7fb7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:22.442069 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c99d41-28ac-4b3f-85aa-305d36db7fb7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:22.443696 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c99d41-28ac-4b3f-85aa-305d36db7fb7 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:22.464436 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 885/1724] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:20:22 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 266 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:20:22.466476 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d1b5da-f177-460c-8937-8ed69849e2c9 service neutron] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:22.484113 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d1b5da-f177-460c-8937-8ed69849e2c9 service neutron] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:22.490559 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 840/1725] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Sep 23 15:20:22 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:20:22.490735 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466b91b5-2fe7-4abc-bd0b-fa15f48049ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:22.491668 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466b91b5-2fe7-4abc-bd0b-fa15f48049ec admin admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:22.492062 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c99d41-28ac-4b3f-85aa-305d36db7fb7 admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:22.492171 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466b91b5-2fe7-4abc-bd0b-fa15f48049ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:22.492525 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466b91b5-2fe7-4abc-bd0b-fa15f48049ec admin admin] PATH_INFO: `/v3/projects` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:22.493377 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466b91b5-2fe7-4abc-bd0b-fa15f48049ec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:22.496257 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466b91b5-2fe7-4abc-bd0b-fa15f48049ec admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:22.510190 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c99d41-28ac-4b3f-85aa-305d36db7fb7 admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:22.516514 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2953c1-cc31-453d-a3ca-49d0fb36782e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:22.517569 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2953c1-cc31-453d-a3ca-49d0fb36782e admin admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:22.517870 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2953c1-cc31-453d-a3ca-49d0fb36782e admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:22.518289 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2953c1-cc31-453d-a3ca-49d0fb36782e admin admin] PATH_INFO: `/v3/roles` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:22.519675 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2953c1-cc31-453d-a3ca-49d0fb36782e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:22.522383 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2953c1-cc31-453d-a3ca-49d0fb36782e admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:22.545848 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 841/1726] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Sep 23 15:20:22 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:20:22.546944 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2953c1-cc31-453d-a3ca-49d0fb36782e admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:22.564439 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2953c1-cc31-453d-a3ca-49d0fb36782e admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:22.565459 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf05597e-0a28-476f-901d-5905c64dc400 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:22.565998 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf05597e-0a28-476f-901d-5905c64dc400 admin admin] REQUEST_METHOD: `PUT` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:22.566351 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf05597e-0a28-476f-901d-5905c64dc400 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:22.567576 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf05597e-0a28-476f-901d-5905c64dc400 admin admin] PATH_INFO: `/v3/projects/0e22570619c84ce197a7963fc475ce03/users/feed427b531448108698d25262ebc938/roles/af257be2dee54d6b942f58650196582c` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:22.619334 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf05597e-0a28-476f-901d-5905c64dc400 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e22570619c84ce197a7963fc475ce03, user_id=feed427b531448108698d25262ebc938, role_id=af257be2dee54d6b942f58650196582c)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:22.626345 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf05597e-0a28-476f-901d-5905c64dc400 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:22.646182 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 886/1727] 10.222.0.55 () {60 vars in 1153 bytes} [Fri Sep 23 15:20:22 2022] POST /identity/v3/projects => generated 365 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:20:22.646182 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466b91b5-2fe7-4abc-bd0b-fa15f48049ec admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:22.761062 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 842/1728] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Sep 23 15:20:22 2022] PUT /identity/v3/projects/0e22570619c84ce197a7963fc475ce03/users/feed427b531448108698d25262ebc938/roles/af257be2dee54d6b942f58650196582c => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:20:22.762315 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf05597e-0a28-476f-901d-5905c64dc400 admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:22.770030 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-bf05597e-0a28-476f-901d-5905c64dc400 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:20:22.792101 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:20:22.804505 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466b91b5-2fe7-4abc-bd0b-fa15f48049ec admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:22.968639 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46649e9-1b58-4869-a2d4-d2067bae491f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:22.969446 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46649e9-1b58-4869-a2d4-d2067bae491f admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:22.970985 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46649e9-1b58-4869-a2d4-d2067bae491f admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:22.971421 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46649e9-1b58-4869-a2d4-d2067bae491f admin admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:22.972416 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46649e9-1b58-4869-a2d4-d2067bae491f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:22.979134 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46649e9-1b58-4869-a2d4-d2067bae491f admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:22.989176 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf05597e-0a28-476f-901d-5905c64dc400 admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:23.004229 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0607db5-2ec9-4cbe-92a5-c2cd749e7131 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3c6c7a1325a47be924268e1aa1a7a18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:23.004868 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0607db5-2ec9-4cbe-92a5-c2cd749e7131 service nova] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:23.005141 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0607db5-2ec9-4cbe-92a5-c2cd749e7131 service nova] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:23.005578 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0607db5-2ec9-4cbe-92a5-c2cd749e7131 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:23.013658 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 887/1729] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Sep 23 15:20:22 2022] GET /identity/v3/roles => generated 1420 bytes in 370 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:20:23.023104 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46649e9-1b58-4869-a2d4-d2067bae491f admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:23.057675 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46649e9-1b58-4869-a2d4-d2067bae491f admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:23.077548 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3e7374-1be4-4b87-845e-696f729a3f45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:23.078450 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3e7374-1be4-4b87-845e-696f729a3f45 admin admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:23.078743 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3e7374-1be4-4b87-845e-696f729a3f45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:23.079018 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3e7374-1be4-4b87-845e-696f729a3f45 admin admin] PATH_INFO: `/v3/users` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:23.079662 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3e7374-1be4-4b87-845e-696f729a3f45 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:23.082035 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3e7374-1be4-4b87-845e-696f729a3f45 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:23.175340 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0607db5-2ec9-4cbe-92a5-c2cd749e7131 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:23.175340 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0607db5-2ec9-4cbe-92a5-c2cd749e7131 service nova] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:23.194971 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 843/1730] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:20:22 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:20:23.196541 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0dec4c-0bda-4b34-bfdc-5ecd869592c4 None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:23.197162 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0dec4c-0bda-4b34-bfdc-5ecd869592c4 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:23.197413 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0dec4c-0bda-4b34-bfdc-5ecd869592c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:23.239263 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-0f0dec4c-0bda-4b34-bfdc-5ecd869592c4 None None] MFA Rules not processed for user `feed427b531448108698d25262ebc938`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:20:23.244064 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-0f0dec4c-0bda-4b34-bfdc-5ecd869592c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:20:23.347335 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 888/1731] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Sep 23 15:20:23 2022] POST /identity/v3/users => generated 289 bytes in 331 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:20:23.347335 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3e7374-1be4-4b87-845e-696f729a3f45 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:23.379769 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3e7374-1be4-4b87-845e-696f729a3f45 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:23.396612 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b3a695-6350-475a-a9b4-d733abc1b77c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:23.397494 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b3a695-6350-475a-a9b4-d733abc1b77c admin admin] REQUEST_METHOD: `PUT` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:23.397736 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b3a695-6350-475a-a9b4-d733abc1b77c admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:23.398016 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b3a695-6350-475a-a9b4-d733abc1b77c admin admin] PATH_INFO: `/v3/projects/8a7ce67550004fb9baa3fd6eb2d52f32/users/5edaf3c7bbb8440dbe132d9909bf9d65/roles/af257be2dee54d6b942f58650196582c` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:23.484518 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b3a695-6350-475a-a9b4-d733abc1b77c admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a7ce67550004fb9baa3fd6eb2d52f32, user_id=5edaf3c7bbb8440dbe132d9909bf9d65, role_id=af257be2dee54d6b942f58650196582c)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:23.488038 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b3a695-6350-475a-a9b4-d733abc1b77c admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:23.490378 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 844/1732] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:20:23 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:20:23.496811 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0dec4c-0bda-4b34-bfdc-5ecd869592c4 None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:23.517753 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0dec4c-0bda-4b34-bfdc-5ecd869592c4 None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:23.548447 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b311e88-48bc-4163-9c04-c364a5685325 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:23.548447 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b311e88-48bc-4163-9c04-c364a5685325 admin admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:23.548447 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b311e88-48bc-4163-9c04-c364a5685325 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:23.548447 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b311e88-48bc-4163-9c04-c364a5685325 admin admin] PATH_INFO: `/v3/roles` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:23.548865 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b311e88-48bc-4163-9c04-c364a5685325 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:23.548865 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b311e88-48bc-4163-9c04-c364a5685325 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:23.558374 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 845/1733] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Sep 23 15:20:23 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:20:23.559665 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b311e88-48bc-4163-9c04-c364a5685325 admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:23.587375 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b311e88-48bc-4163-9c04-c364a5685325 admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:23.602668 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d76871-f79b-401b-a72b-bc5381faa780 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:23.603367 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d76871-f79b-401b-a72b-bc5381faa780 admin admin] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:23.603715 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d76871-f79b-401b-a72b-bc5381faa780 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:23.604304 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d76871-f79b-401b-a72b-bc5381faa780 admin admin] PATH_INFO: `/v3/projects` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:23.605076 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d76871-f79b-401b-a72b-bc5381faa780 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:23.607635 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d76871-f79b-401b-a72b-bc5381faa780 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:23.721653 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 889/1734] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Sep 23 15:20:23 2022] PUT /identity/v3/projects/8a7ce67550004fb9baa3fd6eb2d52f32/users/5edaf3c7bbb8440dbe132d9909bf9d65/roles/af257be2dee54d6b942f58650196582c => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:20:23.722674 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b3a695-6350-475a-a9b4-d733abc1b77c admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:23.787388 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 846/1735] 10.222.0.55 () {60 vars in 1153 bytes} [Fri Sep 23 15:20:23 2022] POST /identity/v3/projects => generated 373 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:20:23.792662 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9817f9a9-95ea-4ef3-a67c-73b3518c1065 None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:23.799614 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9817f9a9-95ea-4ef3-a67c-73b3518c1065 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:23.800154 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9817f9a9-95ea-4ef3-a67c-73b3518c1065 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:23.904287 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-9817f9a9-95ea-4ef3-a67c-73b3518c1065 None None] MFA Rules not processed for user `5edaf3c7bbb8440dbe132d9909bf9d65`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:20:23.905691 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-9817f9a9-95ea-4ef3-a67c-73b3518c1065 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:20:24.000975 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b3a695-6350-475a-a9b4-d733abc1b77c admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:24.105224 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 847/1736] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:20:23 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 314 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:20:24.110971 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9817f9a9-95ea-4ef3-a67c-73b3518c1065 None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:24.145384 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6c5f9f-c703-4cf8-9b3e-bf8a248286a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:24.145993 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6c5f9f-c703-4cf8-9b3e-bf8a248286a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:24.148183 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6c5f9f-c703-4cf8-9b3e-bf8a248286a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:24.148520 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6c5f9f-c703-4cf8-9b3e-bf8a248286a6 admin admin] PATH_INFO: `/v3/projects/04abaf80888047faad3c392c2d0c08c5/users/519e93f1ea224168a8cfe171ec9b2a28/roles/8e2919e79df040e8926f21607818b58c` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:24.185080 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6c5f9f-c703-4cf8-9b3e-bf8a248286a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=04abaf80888047faad3c392c2d0c08c5, user_id=519e93f1ea224168a8cfe171ec9b2a28, role_id=8e2919e79df040e8926f21607818b58c)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:24.190445 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6c5f9f-c703-4cf8-9b3e-bf8a248286a6 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:24.260710 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9817f9a9-95ea-4ef3-a67c-73b3518c1065 None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:24.267565 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e5b14c-a3a7-4bf4-9687-98d3d484ae66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:24.267565 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e5b14c-a3a7-4bf4-9687-98d3d484ae66 admin admin] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:24.267565 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e5b14c-a3a7-4bf4-9687-98d3d484ae66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:24.268286 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e5b14c-a3a7-4bf4-9687-98d3d484ae66 admin admin] PATH_INFO: `/v3/users` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:24.269016 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e5b14c-a3a7-4bf4-9687-98d3d484ae66 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:24.271048 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e5b14c-a3a7-4bf4-9687-98d3d484ae66 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:24.365216 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 890/1737] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Sep 23 15:20:23 2022] PUT /identity/v3/projects/04abaf80888047faad3c392c2d0c08c5/users/519e93f1ea224168a8cfe171ec9b2a28/roles/8e2919e79df040e8926f21607818b58c => generated 0 bytes in 640 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:20:24.366706 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6c5f9f-c703-4cf8-9b3e-bf8a248286a6 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:24.415602 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 848/1738] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Sep 23 15:20:24 2022] POST /identity/v3/users => generated 293 bytes in 310 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:20:24.420968 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e5b14c-a3a7-4bf4-9687-98d3d484ae66 admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:24.465091 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e5b14c-a3a7-4bf4-9687-98d3d484ae66 admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:24.493595 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f9cd62-9ecd-410a-a6e3-be78e951e0a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:24.494238 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f9cd62-9ecd-410a-a6e3-be78e951e0a7 admin admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:24.502341 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6c5f9f-c703-4cf8-9b3e-bf8a248286a6 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:24.503382 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f9cd62-9ecd-410a-a6e3-be78e951e0a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:24.503675 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f9cd62-9ecd-410a-a6e3-be78e951e0a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:24.504436 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f9cd62-9ecd-410a-a6e3-be78e951e0a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:24.508012 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f9cd62-9ecd-410a-a6e3-be78e951e0a7 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:24.523274 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ff593c-3faa-4cd6-8a9e-80d8020b7d3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:24.525102 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ff593c-3faa-4cd6-8a9e-80d8020b7d3f admin admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:24.525481 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ff593c-3faa-4cd6-8a9e-80d8020b7d3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:24.537979 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ff593c-3faa-4cd6-8a9e-80d8020b7d3f admin admin] PATH_INFO: `/v3/projects` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:24.537979 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ff593c-3faa-4cd6-8a9e-80d8020b7d3f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:24.537979 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ff593c-3faa-4cd6-8a9e-80d8020b7d3f admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:24.545943 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 849/1739] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Sep 23 15:20:24 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:20:24.546665 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f9cd62-9ecd-410a-a6e3-be78e951e0a7 admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:24.577354 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f9cd62-9ecd-410a-a6e3-be78e951e0a7 admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:24.618795 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1edd0f1-a980-41dc-b7d1-2617a408284d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:24.619427 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1edd0f1-a980-41dc-b7d1-2617a408284d admin admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:24.619709 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1edd0f1-a980-41dc-b7d1-2617a408284d admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:24.620030 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1edd0f1-a980-41dc-b7d1-2617a408284d admin admin] PATH_INFO: `/v3/roles` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:24.620660 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1edd0f1-a980-41dc-b7d1-2617a408284d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:24.622445 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1edd0f1-a980-41dc-b7d1-2617a408284d admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:24.671722 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 850/1740] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Sep 23 15:20:24 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:20:24.671722 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1edd0f1-a980-41dc-b7d1-2617a408284d admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:24.691620 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1edd0f1-a980-41dc-b7d1-2617a408284d admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:24.705379 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2400db6d-c6c3-48ef-985a-8ce6d20df369 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:24.706804 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2400db6d-c6c3-48ef-985a-8ce6d20df369 admin admin] REQUEST_METHOD: `PUT` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:24.717554 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2400db6d-c6c3-48ef-985a-8ce6d20df369 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:24.717867 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2400db6d-c6c3-48ef-985a-8ce6d20df369 admin admin] PATH_INFO: `/v3/projects/04abaf80888047faad3c392c2d0c08c5/users/519e93f1ea224168a8cfe171ec9b2a28/roles/8e2919e79df040e8926f21607818b58c` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:24.721399 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2400db6d-c6c3-48ef-985a-8ce6d20df369 admin admin] RBAC: Authorizing `identity:create_grant(project_id=04abaf80888047faad3c392c2d0c08c5, user_id=519e93f1ea224168a8cfe171ec9b2a28, role_id=8e2919e79df040e8926f21607818b58c)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:24.724213 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2400db6d-c6c3-48ef-985a-8ce6d20df369 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:24.740142 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 851/1741] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Sep 23 15:20:24 2022] PUT /identity/v3/projects/04abaf80888047faad3c392c2d0c08c5/users/519e93f1ea224168a8cfe171ec9b2a28/roles/8e2919e79df040e8926f21607818b58c => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:20:24.740142 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2400db6d-c6c3-48ef-985a-8ce6d20df369 admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:24.759005 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 891/1742] 10.222.0.55 () {60 vars in 1153 bytes} [Fri Sep 23 15:20:24 2022] POST /identity/v3/projects => generated 367 bytes in 393 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:20:24.761071 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70328edd-93ff-415e-b441-bf0ca6e01dff None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:24.761516 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70328edd-93ff-415e-b441-bf0ca6e01dff None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:24.761793 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70328edd-93ff-415e-b441-bf0ca6e01dff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:24.820443 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-70328edd-93ff-415e-b441-bf0ca6e01dff None None] MFA Rules not processed for user `519e93f1ea224168a8cfe171ec9b2a28`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:20:24.823870 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-70328edd-93ff-415e-b441-bf0ca6e01dff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:20:24.941698 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2400db6d-c6c3-48ef-985a-8ce6d20df369 admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:25.130920 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 892/1743] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:20:24 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 371 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:20:25.148980 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70328edd-93ff-415e-b441-bf0ca6e01dff None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:25.158258 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff7c7fa-fc0e-4352-990f-b2a059a3023a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:25.171157 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70328edd-93ff-415e-b441-bf0ca6e01dff None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:25.176223 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff7c7fa-fc0e-4352-990f-b2a059a3023a admin admin] REQUEST_METHOD: `PUT` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:25.176662 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff7c7fa-fc0e-4352-990f-b2a059a3023a admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:25.179288 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f60524-8b8e-4506-88e7-50a44bcd24cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:25.179763 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff7c7fa-fc0e-4352-990f-b2a059a3023a admin admin] PATH_INFO: `/v3/projects/1d9efe9104fd42498faab3f108aac45f/users/4db53ae698d94e70a1a98af04e751fb6/roles/8e2919e79df040e8926f21607818b58c` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:25.180335 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f60524-8b8e-4506-88e7-50a44bcd24cd admin admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:25.180791 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f60524-8b8e-4506-88e7-50a44bcd24cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:25.181410 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f60524-8b8e-4506-88e7-50a44bcd24cd admin admin] PATH_INFO: `/v3/users` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:25.182243 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f60524-8b8e-4506-88e7-50a44bcd24cd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:25.185475 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f60524-8b8e-4506-88e7-50a44bcd24cd admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:25.250661 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff7c7fa-fc0e-4352-990f-b2a059a3023a admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d9efe9104fd42498faab3f108aac45f, user_id=4db53ae698d94e70a1a98af04e751fb6, role_id=8e2919e79df040e8926f21607818b58c)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:25.254533 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff7c7fa-fc0e-4352-990f-b2a059a3023a admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:25.332609 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 893/1744] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Sep 23 15:20:25 2022] POST /identity/v3/users => generated 290 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:20:25.332609 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f60524-8b8e-4506-88e7-50a44bcd24cd admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:25.336159 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-d5f60524-8b8e-4506-88e7-50a44bcd24cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:20:25.380667 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 852/1745] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Sep 23 15:20:24 2022] PUT /identity/v3/projects/1d9efe9104fd42498faab3f108aac45f/users/4db53ae698d94e70a1a98af04e751fb6/roles/8e2919e79df040e8926f21607818b58c => generated 0 bytes in 636 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:20:25.380667 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff7c7fa-fc0e-4352-990f-b2a059a3023a admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:25.587484 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f60524-8b8e-4506-88e7-50a44bcd24cd admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:25.689025 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff7c7fa-fc0e-4352-990f-b2a059a3023a admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:25.703580 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b7df99-4654-41fb-9438-2280ba73ac32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:25.704563 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b7df99-4654-41fb-9438-2280ba73ac32 admin admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:25.705063 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b7df99-4654-41fb-9438-2280ba73ac32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:25.706141 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b7df99-4654-41fb-9438-2280ba73ac32 admin admin] PATH_INFO: `/v3/roles` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:25.706995 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b7df99-4654-41fb-9438-2280ba73ac32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:25.709532 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b7df99-4654-41fb-9438-2280ba73ac32 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:25.722292 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 853/1746] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Sep 23 15:20:25 2022] GET /identity/v3/roles => generated 1420 bytes in 347 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:20:25.723641 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b7df99-4654-41fb-9438-2280ba73ac32 admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:25.734504 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eee728c-8822-469b-b4d1-f17fc18c3eb7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feca6278be904ef8b0dc580a67e477f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:25.734504 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eee728c-8822-469b-b4d1-f17fc18c3eb7 service cinder] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:25.734504 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eee728c-8822-469b-b4d1-f17fc18c3eb7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:25.734504 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eee728c-8822-469b-b4d1-f17fc18c3eb7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:25.735444 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b7df99-4654-41fb-9438-2280ba73ac32 admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:25.742008 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b49043-ecbd-4c6b-b80d-159c0cab16ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:25.742736 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b49043-ecbd-4c6b-b80d-159c0cab16ad admin admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:25.743022 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b49043-ecbd-4c6b-b80d-159c0cab16ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:25.743499 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b49043-ecbd-4c6b-b80d-159c0cab16ad admin admin] PATH_INFO: `/v3/roles` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:25.744250 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b49043-ecbd-4c6b-b80d-159c0cab16ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:25.747535 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b49043-ecbd-4c6b-b80d-159c0cab16ad admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:25.765550 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 854/1747] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Sep 23 15:20:25 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:20:25.765550 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b49043-ecbd-4c6b-b80d-159c0cab16ad admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:25.776594 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b49043-ecbd-4c6b-b80d-159c0cab16ad admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:25.782952 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0844bf1c-43ee-442f-9c8a-41edf003180e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:25.783681 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0844bf1c-43ee-442f-9c8a-41edf003180e admin admin] REQUEST_METHOD: `PUT` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:25.783969 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0844bf1c-43ee-442f-9c8a-41edf003180e admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:25.784312 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0844bf1c-43ee-442f-9c8a-41edf003180e admin admin] PATH_INFO: `/v3/projects/885bc6c7a0914f41b8c8a078748c64de/users/3167e37d05d74802a3215b177b42226e/roles/8e2919e79df040e8926f21607818b58c` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:25.844892 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0844bf1c-43ee-442f-9c8a-41edf003180e admin admin] RBAC: Authorizing `identity:create_grant(project_id=885bc6c7a0914f41b8c8a078748c64de, user_id=3167e37d05d74802a3215b177b42226e, role_id=8e2919e79df040e8926f21607818b58c)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:25.858681 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0844bf1c-43ee-442f-9c8a-41edf003180e admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:25.947777 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eee728c-8822-469b-b4d1-f17fc18c3eb7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:25.947777 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eee728c-8822-469b-b4d1-f17fc18c3eb7 service cinder] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:25.970161 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 894/1748] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:20:25 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 638 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:20:25.971230 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eee728c-8822-469b-b4d1-f17fc18c3eb7 service cinder] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:26.002723 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eee728c-8822-469b-b4d1-f17fc18c3eb7 service cinder] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:26.018560 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 855/1749] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Sep 23 15:20:25 2022] PUT /identity/v3/projects/885bc6c7a0914f41b8c8a078748c64de/users/3167e37d05d74802a3215b177b42226e/roles/8e2919e79df040e8926f21607818b58c => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:20:26.023703 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0844bf1c-43ee-442f-9c8a-41edf003180e admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:26.145788 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063e5954-d83d-454d-84aa-a860eb79672b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:26.146705 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063e5954-d83d-454d-84aa-a860eb79672b admin admin] REQUEST_METHOD: `PUT` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:26.146988 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063e5954-d83d-454d-84aa-a860eb79672b admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:26.147263 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063e5954-d83d-454d-84aa-a860eb79672b admin admin] PATH_INFO: `/v3/projects/1d9efe9104fd42498faab3f108aac45f/users/4db53ae698d94e70a1a98af04e751fb6/roles/8e2919e79df040e8926f21607818b58c` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:26.157939 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063e5954-d83d-454d-84aa-a860eb79672b admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d9efe9104fd42498faab3f108aac45f, user_id=4db53ae698d94e70a1a98af04e751fb6, role_id=8e2919e79df040e8926f21607818b58c)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:26.160904 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063e5954-d83d-454d-84aa-a860eb79672b admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:26.197871 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 895/1750] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Sep 23 15:20:25 2022] PUT /identity/v3/projects/1d9efe9104fd42498faab3f108aac45f/users/4db53ae698d94e70a1a98af04e751fb6/roles/8e2919e79df040e8926f21607818b58c => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:20:26.210350 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ec5ca6-e764-4cdb-b088-68f99d8b0c81 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:26.210661 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ec5ca6-e764-4cdb-b088-68f99d8b0c81 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:26.210948 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ec5ca6-e764-4cdb-b088-68f99d8b0c81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:26.220872 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:20:26.228807 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0844bf1c-43ee-442f-9c8a-41edf003180e admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:26.283494 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-d6ec5ca6-e764-4cdb-b088-68f99d8b0c81 None None] MFA Rules not processed for user `4db53ae698d94e70a1a98af04e751fb6`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:20:26.284814 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-d6ec5ca6-e764-4cdb-b088-68f99d8b0c81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:20:26.366017 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97b3767-f2f5-439f-88a1-cd9acbcdde51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:26.368654 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97b3767-f2f5-439f-88a1-cd9acbcdde51 admin admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:26.369153 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97b3767-f2f5-439f-88a1-cd9acbcdde51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:26.370014 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97b3767-f2f5-439f-88a1-cd9acbcdde51 admin admin] PATH_INFO: `/v3/roles` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:26.376618 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97b3767-f2f5-439f-88a1-cd9acbcdde51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:26.387074 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97b3767-f2f5-439f-88a1-cd9acbcdde51 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:26.459610 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 856/1751] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Sep 23 15:20:26 2022] GET /identity/v3/roles => generated 1420 bytes in 437 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:20:26.461562 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97b3767-f2f5-439f-88a1-cd9acbcdde51 admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:26.494872 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 896/1752] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:20:26 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:20:26.496196 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ec5ca6-e764-4cdb-b088-68f99d8b0c81 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:26.508886 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ec5ca6-e764-4cdb-b088-68f99d8b0c81 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:26.513974 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184ea959-d83d-4ab9-a8a7-0da6af1e61ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:26.516161 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184ea959-d83d-4ab9-a8a7-0da6af1e61ca admin admin] REQUEST_METHOD: `PUT` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:26.516515 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184ea959-d83d-4ab9-a8a7-0da6af1e61ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:26.516750 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184ea959-d83d-4ab9-a8a7-0da6af1e61ca admin admin] PATH_INFO: `/v3/projects/885bc6c7a0914f41b8c8a078748c64de/users/3167e37d05d74802a3215b177b42226e/roles/8e2919e79df040e8926f21607818b58c` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:26.519555 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184ea959-d83d-4ab9-a8a7-0da6af1e61ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=885bc6c7a0914f41b8c8a078748c64de, user_id=3167e37d05d74802a3215b177b42226e, role_id=8e2919e79df040e8926f21607818b58c)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:26.522644 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184ea959-d83d-4ab9-a8a7-0da6af1e61ca admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:26.537357 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 897/1753] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Sep 23 15:20:26 2022] PUT /identity/v3/projects/885bc6c7a0914f41b8c8a078748c64de/users/3167e37d05d74802a3215b177b42226e/roles/8e2919e79df040e8926f21607818b58c => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:20:26.551059 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ac3092-9e3b-42e2-b05d-49939cf06039 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:26.551302 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ac3092-9e3b-42e2-b05d-49939cf06039 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:26.551571 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ac3092-9e3b-42e2-b05d-49939cf06039 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:26.603533 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-54ac3092-9e3b-42e2-b05d-49939cf06039 None None] MFA Rules not processed for user `3167e37d05d74802a3215b177b42226e`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:20:26.604944 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-54ac3092-9e3b-42e2-b05d-49939cf06039 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:20:26.634179 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97b3767-f2f5-439f-88a1-cd9acbcdde51 admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:26.735111 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3442202e-7bbb-4a06-9970-215b7f65f382 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feca6278be904ef8b0dc580a67e477f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:26.735869 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3442202e-7bbb-4a06-9970-215b7f65f382 service cinder] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:26.736250 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3442202e-7bbb-4a06-9970-215b7f65f382 service cinder] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:26.736551 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3442202e-7bbb-4a06-9970-215b7f65f382 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:26.781016 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 898/1754] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:20:26 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 15:20:26.852481 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3442202e-7bbb-4a06-9970-215b7f65f382 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:26.854359 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3442202e-7bbb-4a06-9970-215b7f65f382 service cinder] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:26.876921 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 857/1755] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:20:26 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:20:26.939616 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784d0af8-5fee-4bf8-bbe7-06b1781d2f73 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:26.940001 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784d0af8-5fee-4bf8-bbe7-06b1781d2f73 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:26.940354 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784d0af8-5fee-4bf8-bbe7-06b1781d2f73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:26.993884 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-784d0af8-5fee-4bf8-bbe7-06b1781d2f73 None None] MFA Rules not processed for user `42dc378333ae4bbdbd42ce041ba15d3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:20:26.999443 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-784d0af8-5fee-4bf8-bbe7-06b1781d2f73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:20:27.005364 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:20:27.061028 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3442202e-7bbb-4a06-9970-215b7f65f382 service cinder] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:27.067721 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-3442202e-7bbb-4a06-9970-215b7f65f382 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:20:27.085123 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3442202e-7bbb-4a06-9970-215b7f65f382 service cinder] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:27.089240 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e97a0cf-3546-4a22-96c3-72031d519d4b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feca6278be904ef8b0dc580a67e477f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:27.092276 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e97a0cf-3546-4a22-96c3-72031d519d4b service cinder] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:27.092276 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e97a0cf-3546-4a22-96c3-72031d519d4b service cinder] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:27.092276 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e97a0cf-3546-4a22-96c3-72031d519d4b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:27.099654 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 899/1756] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:20:26 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:20:27.227739 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e97a0cf-3546-4a22-96c3-72031d519d4b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:27.229871 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e97a0cf-3546-4a22-96c3-72031d519d4b service cinder] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:27.253479 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 858/1757] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:20:27 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:20:27.460354 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784d0af8-5fee-4bf8-bbe7-06b1781d2f73 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:27.477420 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784d0af8-5fee-4bf8-bbe7-06b1781d2f73 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:27.487469 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5947f1-fc87-4390-8ad2-abb741c54d14 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feca6278be904ef8b0dc580a67e477f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:27.488283 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5947f1-fc87-4390-8ad2-abb741c54d14 service cinder] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:27.488576 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5947f1-fc87-4390-8ad2-abb741c54d14 service cinder] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:27.488917 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5947f1-fc87-4390-8ad2-abb741c54d14 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:27.512657 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5947f1-fc87-4390-8ad2-abb741c54d14 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:27.514250 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5947f1-fc87-4390-8ad2-abb741c54d14 service cinder] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:27.541105 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 900/1758] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:20:27 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:20:27.623328 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49aadc7-71dc-4d4c-a182-8c605de93fb6 None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:27.623928 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49aadc7-71dc-4d4c-a182-8c605de93fb6 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:27.623928 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49aadc7-71dc-4d4c-a182-8c605de93fb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:27.685912 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-f49aadc7-71dc-4d4c-a182-8c605de93fb6 None None] MFA Rules not processed for user `68206a6b4f244d45bf7bf1504396e430`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:20:27.687274 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-f49aadc7-71dc-4d4c-a182-8c605de93fb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:20:27.768043 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 859/1759] 10.222.0.55 () {58 vars in 1009 bytes} [Fri Sep 23 15:20:27 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:20:27.978077 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5947f1-fc87-4390-8ad2-abb741c54d14 service cinder] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:27.988397 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5947f1-fc87-4390-8ad2-abb741c54d14 service cinder] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:28.007845 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c9a6c4-091a-4fb2-9749-3cf0a1a8bdf1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feca6278be904ef8b0dc580a67e477f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:28.008800 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c9a6c4-091a-4fb2-9749-3cf0a1a8bdf1 service cinder] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:28.009155 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c9a6c4-091a-4fb2-9749-3cf0a1a8bdf1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:28.009530 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c9a6c4-091a-4fb2-9749-3cf0a1a8bdf1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:28.091639 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c9a6c4-091a-4fb2-9749-3cf0a1a8bdf1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:28.093259 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c9a6c4-091a-4fb2-9749-3cf0a1a8bdf1 service cinder] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:28.102407 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 901/1760] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:20:27 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:20:28.613277 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49aadc7-71dc-4d4c-a182-8c605de93fb6 None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:28.614836 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-f49aadc7-71dc-4d4c-a182-8c605de93fb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:20:28.936257 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49aadc7-71dc-4d4c-a182-8c605de93fb6 None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:29.035545 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27fe66f-98b8-4a72-8376-ffeff778be52 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2228c0b49a0249bbaf524a8c4e9e108b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:29.035545 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27fe66f-98b8-4a72-8376-ffeff778be52 service swift] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:29.035545 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27fe66f-98b8-4a72-8376-ffeff778be52 service swift] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:29.035545 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27fe66f-98b8-4a72-8376-ffeff778be52 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:29.038345 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27fe66f-98b8-4a72-8376-ffeff778be52 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:29.038345 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27fe66f-98b8-4a72-8376-ffeff778be52 service swift] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:29.038345 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 860/1761] 10.222.0.55 () {60 vars in 1384 bytes} [Fri Sep 23 15:20:28 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 23 15:20:29.653343 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4c6636-77ed-4b63-acf8-814acd0b7126 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:29.653960 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4c6636-77ed-4b63-acf8-814acd0b7126 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:29.654118 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4c6636-77ed-4b63-acf8-814acd0b7126 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:29.695723 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-1c4c6636-77ed-4b63-acf8-814acd0b7126 None None] MFA Rules not processed for user `d259178570864ffeb936ee10dfec6b28`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:20:29.697236 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-1c4c6636-77ed-4b63-acf8-814acd0b7126 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:20:29.808812 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 902/1762] 10.222.0.77 () {58 vars in 1015 bytes} [Fri Sep 23 15:20:29 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:20:29.998917 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27fe66f-98b8-4a72-8376-ffeff778be52 service swift] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:30.002314 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-d27fe66f-98b8-4a72-8376-ffeff778be52 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:20:30.044511 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27fe66f-98b8-4a72-8376-ffeff778be52 service swift] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:30.070001 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70bb44f-dfe7-4d89-8b4f-0cdc8c680027 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd259178570864ffeb936ee10dfec6b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:30.070638 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70bb44f-dfe7-4d89-8b4f-0cdc8c680027 service neutron] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:30.070924 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70bb44f-dfe7-4d89-8b4f-0cdc8c680027 service neutron] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:30.071205 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70bb44f-dfe7-4d89-8b4f-0cdc8c680027 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:30.121862 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70bb44f-dfe7-4d89-8b4f-0cdc8c680027 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:30.121862 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70bb44f-dfe7-4d89-8b4f-0cdc8c680027 service neutron] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:30.147883 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 861/1763] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:20:29 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:20:32.391940 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4c6636-77ed-4b63-acf8-814acd0b7126 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:32.593507 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4c6636-77ed-4b63-acf8-814acd0b7126 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:32.608490 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaafa0e8-c722-46ef-97eb-242dea5c0e0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:32.612690 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaafa0e8-c722-46ef-97eb-242dea5c0e0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:32.613239 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaafa0e8-c722-46ef-97eb-242dea5c0e0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:32.613672 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaafa0e8-c722-46ef-97eb-242dea5c0e0e admin admin] PATH_INFO: `/v3/users/feed427b531448108698d25262ebc938` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:32.617639 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaafa0e8-c722-46ef-97eb-242dea5c0e0e admin admin] RBAC: Authorizing `identity:delete_user(user_id=feed427b531448108698d25262ebc938)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:32.622382 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaafa0e8-c722-46ef-97eb-242dea5c0e0e admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:33.658229 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:20:33.694063 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-aaafa0e8-c722-46ef-97eb-242dea5c0e0e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'feed427b531448108698d25262ebc938'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:33.713555 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-aaafa0e8-c722-46ef-97eb-242dea5c0e0e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'feed427b531448108698d25262ebc938'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:33.730645 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-aaafa0e8-c722-46ef-97eb-242dea5c0e0e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'feed427b531448108698d25262ebc938'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:33.861178 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-aaafa0e8-c722-46ef-97eb-242dea5c0e0e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'feed427b531448108698d25262ebc938'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:33.863848 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-aaafa0e8-c722-46ef-97eb-242dea5c0e0e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'feed427b531448108698d25262ebc938'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:33.869787 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e97edc-b8a5-418a-97e8-dc084bca98c1 None None] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:33.872618 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 903/1764] 10.222.0.55 () {58 vars in 1224 bytes} [Fri Sep 23 15:20:32 2022] DELETE /identity/v3/users/feed427b531448108698d25262ebc938 => generated 0 bytes in 1487 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:20:33.873386 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e97edc-b8a5-418a-97e8-dc084bca98c1 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:33.873999 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e97edc-b8a5-418a-97e8-dc084bca98c1 None None] PATH_INFO: `/` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:33.878651 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 862/1765] 10.222.0.55 () {54 vars in 921 bytes} [Fri Sep 23 15:20:33 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:20:33.882761 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9a40d6-3566-4492-bc50-e40017359267 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:33.883058 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9a40d6-3566-4492-bc50-e40017359267 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:33.883347 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9a40d6-3566-4492-bc50-e40017359267 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:33.895424 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86dd9e9-1808-46e2-a132-a304e109e81e None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:33.896093 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86dd9e9-1808-46e2-a132-a304e109e81e None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:33.896730 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86dd9e9-1808-46e2-a132-a304e109e81e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:33.931636 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-2d9a40d6-3566-4492-bc50-e40017359267 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:20:33.934934 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-2d9a40d6-3566-4492-bc50-e40017359267 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:20:33.938422 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-f86dd9e9-1808-46e2-a132-a304e109e81e None None] MFA Rules not processed for user `b3c6c7a1325a47be924268e1aa1a7a18`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:20:33.939713 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-f86dd9e9-1808-46e2-a132-a304e109e81e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:20:34.129622 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 863/1766] 10.222.0.55 () {58 vars in 1018 bytes} [Fri Sep 23 15:20:33 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 15:20:34.135279 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 904/1767] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:20:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:20:34.227207 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86dd9e9-1808-46e2-a132-a304e109e81e None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:34.229149 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-f86dd9e9-1808-46e2-a132-a304e109e81e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:20:34.399336 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86dd9e9-1808-46e2-a132-a304e109e81e None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:34.407735 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8781749d-16ac-41c5-8f66-b8bbb2bd680f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd259178570864ffeb936ee10dfec6b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:34.411267 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8781749d-16ac-41c5-8f66-b8bbb2bd680f service neutron] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:34.411561 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8781749d-16ac-41c5-8f66-b8bbb2bd680f service neutron] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:34.411840 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8781749d-16ac-41c5-8f66-b8bbb2bd680f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:34.450439 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8781749d-16ac-41c5-8f66-b8bbb2bd680f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:34.452635 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8781749d-16ac-41c5-8f66-b8bbb2bd680f service neutron] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:34.478732 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 864/1768] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:20:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 252 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:20:34.929583 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9a40d6-3566-4492-bc50-e40017359267 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:34.931879 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-2d9a40d6-3566-4492-bc50-e40017359267 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:20:34.966092 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9a40d6-3566-4492-bc50-e40017359267 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:34.975938 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d168d06-06cd-4e3c-ab91-7b6d219a3908 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3c6c7a1325a47be924268e1aa1a7a18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:34.978881 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d168d06-06cd-4e3c-ab91-7b6d219a3908 service nova] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:34.979405 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d168d06-06cd-4e3c-ab91-7b6d219a3908 service nova] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:34.980335 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d168d06-06cd-4e3c-ab91-7b6d219a3908 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:34.997948 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d168d06-06cd-4e3c-ab91-7b6d219a3908 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:35.000666 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d168d06-06cd-4e3c-ab91-7b6d219a3908 service nova] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:35.025936 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 905/1769] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:20:34 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 23 15:20:36.532227 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:20:36.701692 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8781749d-16ac-41c5-8f66-b8bbb2bd680f service neutron] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:36.703240 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-8781749d-16ac-41c5-8f66-b8bbb2bd680f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:20:36.725525 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8781749d-16ac-41c5-8f66-b8bbb2bd680f service neutron] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:36.751346 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f97f676-6686-4914-986a-dd76a91aa532 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:36.752150 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f97f676-6686-4914-986a-dd76a91aa532 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:36.752326 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f97f676-6686-4914-986a-dd76a91aa532 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:36.752594 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f97f676-6686-4914-986a-dd76a91aa532 admin admin] PATH_INFO: `/v3/projects/0e22570619c84ce197a7963fc475ce03` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:36.756980 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f97f676-6686-4914-986a-dd76a91aa532 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0e22570619c84ce197a7963fc475ce03)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:36.759145 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f97f676-6686-4914-986a-dd76a91aa532 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:36.963787 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dacf42b-ae4a-4d57-ab9b-8538cba2a511 None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:36.965871 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dacf42b-ae4a-4d57-ab9b-8538cba2a511 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:36.966324 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dacf42b-ae4a-4d57-ab9b-8538cba2a511 None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:36.968256 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 906/1770] 10.222.0.55 () {54 vars in 921 bytes} [Fri Sep 23 15:20:36 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:20:37.113582 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-8f97f676-6686-4914-986a-dd76a91aa532 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0e22570619c84ce197a7963fc475ce03', 'request_id': 'req-8f97f676-6686-4914-986a-dd76a91aa532'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:37.141980 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-8f97f676-6686-4914-986a-dd76a91aa532 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0e22570619c84ce197a7963fc475ce03', 'request_id': 'req-8f97f676-6686-4914-986a-dd76a91aa532'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:37.146109 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c01e6f-a609-4a30-bd9e-286df0cdd813 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:37.146491 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c01e6f-a609-4a30-bd9e-286df0cdd813 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:37.146893 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c01e6f-a609-4a30-bd9e-286df0cdd813 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:37.205034 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-09c01e6f-a609-4a30-bd9e-286df0cdd813 None None] MFA Rules not processed for user `b3c6c7a1325a47be924268e1aa1a7a18`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:20:37.206684 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-09c01e6f-a609-4a30-bd9e-286df0cdd813 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:20:37.425360 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 907/1771] 10.222.0.55 () {58 vars in 1018 bytes} [Fri Sep 23 15:20:37 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 15:20:37.766941 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c01e6f-a609-4a30-bd9e-286df0cdd813 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:37.783442 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c01e6f-a609-4a30-bd9e-286df0cdd813 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:37.791497 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b2adb9-2e1c-4e53-bdb5-4d134194009c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3c6c7a1325a47be924268e1aa1a7a18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:37.793786 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b2adb9-2e1c-4e53-bdb5-4d134194009c service nova] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:37.794474 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b2adb9-2e1c-4e53-bdb5-4d134194009c service nova] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:37.795147 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b2adb9-2e1c-4e53-bdb5-4d134194009c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:37.810214 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b2adb9-2e1c-4e53-bdb5-4d134194009c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:37.812074 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b2adb9-2e1c-4e53-bdb5-4d134194009c service nova] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:37.836440 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 908/1772] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:20:37 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 23 15:20:37.918302 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-8f97f676-6686-4914-986a-dd76a91aa532 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0e22570619c84ce197a7963fc475ce03', 'request_id': 'req-8f97f676-6686-4914-986a-dd76a91aa532'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:37.938076 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-8f97f676-6686-4914-986a-dd76a91aa532 admin admin] The token cache is being invalidate because project 0e22570619c84ce197a7963fc475ce03 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81993) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 15:20:37.939545 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-8f97f676-6686-4914-986a-dd76a91aa532 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:37.940671 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-8f97f676-6686-4914-986a-dd76a91aa532 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:37.946291 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 865/1773] 10.222.0.55 () {58 vars in 1233 bytes} [Fri Sep 23 15:20:36 2022] DELETE /identity/v3/projects/0e22570619c84ce197a7963fc475ce03 => generated 0 bytes in 1245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:20:37.956121 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b2adb9-2e1c-4e53-bdb5-4d134194009c service nova] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:38.084608 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b2adb9-2e1c-4e53-bdb5-4d134194009c service nova] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:38.104021 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39146f27-567a-461b-b932-5abbf143aa30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:38.104021 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39146f27-567a-461b-b932-5abbf143aa30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:38.104021 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39146f27-567a-461b-b932-5abbf143aa30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:38.104021 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39146f27-567a-461b-b932-5abbf143aa30 admin admin] PATH_INFO: `/v3/users/4db53ae698d94e70a1a98af04e751fb6` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:38.106658 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39146f27-567a-461b-b932-5abbf143aa30 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4db53ae698d94e70a1a98af04e751fb6)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:38.112803 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39146f27-567a-461b-b932-5abbf143aa30 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:38.982259 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-39146f27-567a-461b-b932-5abbf143aa30 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4db53ae698d94e70a1a98af04e751fb6'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:39.025926 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-39146f27-567a-461b-b932-5abbf143aa30 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4db53ae698d94e70a1a98af04e751fb6'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:39.046957 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-39146f27-567a-461b-b932-5abbf143aa30 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4db53ae698d94e70a1a98af04e751fb6'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:39.270184 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-39146f27-567a-461b-b932-5abbf143aa30 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4db53ae698d94e70a1a98af04e751fb6'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:39.270840 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-39146f27-567a-461b-b932-5abbf143aa30 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4db53ae698d94e70a1a98af04e751fb6'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:39.279372 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 909/1774] 10.222.0.55 () {58 vars in 1224 bytes} [Fri Sep 23 15:20:37 2022] DELETE /identity/v3/users/4db53ae698d94e70a1a98af04e751fb6 => generated 0 bytes in 1325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:20:40.234410 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f97f676-6686-4914-986a-dd76a91aa532 admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:40.238646 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-8f97f676-6686-4914-986a-dd76a91aa532 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:20:40.257515 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f97f676-6686-4914-986a-dd76a91aa532 admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:40.329157 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0115d8ac-28c2-473f-8bc0-4bd6c5c67318 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:40.329937 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0115d8ac-28c2-473f-8bc0-4bd6c5c67318 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:40.330229 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0115d8ac-28c2-473f-8bc0-4bd6c5c67318 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:40.330510 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0115d8ac-28c2-473f-8bc0-4bd6c5c67318 admin admin] PATH_INFO: `/v3/projects/1d9efe9104fd42498faab3f108aac45f` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:40.331842 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0115d8ac-28c2-473f-8bc0-4bd6c5c67318 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1d9efe9104fd42498faab3f108aac45f)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:40.334193 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0115d8ac-28c2-473f-8bc0-4bd6c5c67318 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:40.563736 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-0115d8ac-28c2-473f-8bc0-4bd6c5c67318 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1d9efe9104fd42498faab3f108aac45f', 'request_id': 'req-0115d8ac-28c2-473f-8bc0-4bd6c5c67318'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:40.569295 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-0115d8ac-28c2-473f-8bc0-4bd6c5c67318 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1d9efe9104fd42498faab3f108aac45f', 'request_id': 'req-0115d8ac-28c2-473f-8bc0-4bd6c5c67318'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:40.670074 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-0115d8ac-28c2-473f-8bc0-4bd6c5c67318 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1d9efe9104fd42498faab3f108aac45f', 'request_id': 'req-0115d8ac-28c2-473f-8bc0-4bd6c5c67318'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:40.683669 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-0115d8ac-28c2-473f-8bc0-4bd6c5c67318 admin admin] The token cache is being invalidate because project 1d9efe9104fd42498faab3f108aac45f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81993) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 15:20:40.684057 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-0115d8ac-28c2-473f-8bc0-4bd6c5c67318 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:40.685120 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-0115d8ac-28c2-473f-8bc0-4bd6c5c67318 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:40.686698 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 866/1775] 10.222.0.55 () {58 vars in 1233 bytes} [Fri Sep 23 15:20:40 2022] DELETE /identity/v3/projects/1d9efe9104fd42498faab3f108aac45f => generated 0 bytes in 454 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 15:20:40.691937 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e190171-f002-4c09-9702-5fe23eeab5f8 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:40.692496 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e190171-f002-4c09-9702-5fe23eeab5f8 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:40.692595 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e190171-f002-4c09-9702-5fe23eeab5f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:40.714070 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-8e190171-f002-4c09-9702-5fe23eeab5f8 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:20:40.715312 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-8e190171-f002-4c09-9702-5fe23eeab5f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:20:40.799346 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 910/1776] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:20:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 15:20:40.807251 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e46d993-93a2-4ba2-9136-e7cd29d70276 None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:40.807540 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e46d993-93a2-4ba2-9136-e7cd29d70276 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:40.808515 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e46d993-93a2-4ba2-9136-e7cd29d70276 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:40.828976 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-5e46d993-93a2-4ba2-9136-e7cd29d70276 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:20:40.830292 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-5e46d993-93a2-4ba2-9136-e7cd29d70276 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:20:40.838673 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 867/1777] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:20:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:20:40.843331 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e190171-f002-4c09-9702-5fe23eeab5f8 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:40.853040 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e190171-f002-4c09-9702-5fe23eeab5f8 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:40.858627 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fed0b33-3f61-4e6a-a49d-95f37a9434cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:40.859239 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fed0b33-3f61-4e6a-a49d-95f37a9434cb admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:40.863285 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fed0b33-3f61-4e6a-a49d-95f37a9434cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:40.863285 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fed0b33-3f61-4e6a-a49d-95f37a9434cb admin admin] PATH_INFO: `/v3/domains` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:40.863285 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fed0b33-3f61-4e6a-a49d-95f37a9434cb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:20:40.863285 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fed0b33-3f61-4e6a-a49d-95f37a9434cb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:40.863285 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fed0b33-3f61-4e6a-a49d-95f37a9434cb admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:40.878908 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 911/1778] 10.222.0.55 () {58 vars in 1153 bytes} [Fri Sep 23 15:20:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 15:20:40.880233 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e46d993-93a2-4ba2-9136-e7cd29d70276 None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:40.889505 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e46d993-93a2-4ba2-9136-e7cd29d70276 None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:40.894857 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df05ee28-fb0d-4f7a-bf5e-0e6f99af92c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:40.895483 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df05ee28-fb0d-4f7a-bf5e-0e6f99af92c1 admin admin] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:40.895765 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df05ee28-fb0d-4f7a-bf5e-0e6f99af92c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:40.896077 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df05ee28-fb0d-4f7a-bf5e-0e6f99af92c1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:40.896721 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df05ee28-fb0d-4f7a-bf5e-0e6f99af92c1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:40.898831 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df05ee28-fb0d-4f7a-bf5e-0e6f99af92c1 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:40.955601 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fed0b33-3f61-4e6a-a49d-95f37a9434cb admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:40.957911 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-3fed0b33-3f61-4e6a-a49d-95f37a9434cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:20:40.978185 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fed0b33-3f61-4e6a-a49d-95f37a9434cb admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:40.983889 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575e4bb1-554f-48ba-bfe1-e3914fd6d2ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:40.984622 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575e4bb1-554f-48ba-bfe1-e3914fd6d2ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:40.984914 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575e4bb1-554f-48ba-bfe1-e3914fd6d2ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:40.985350 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575e4bb1-554f-48ba-bfe1-e3914fd6d2ea admin admin] PATH_INFO: `/v3/users/5edaf3c7bbb8440dbe132d9909bf9d65` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:40.986497 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575e4bb1-554f-48ba-bfe1-e3914fd6d2ea admin admin] RBAC: Authorizing `identity:delete_user(user_id=5edaf3c7bbb8440dbe132d9909bf9d65)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:40.988435 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575e4bb1-554f-48ba-bfe1-e3914fd6d2ea admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:41.082160 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 868/1779] 10.222.0.55 () {60 vars in 1153 bytes} [Fri Sep 23 15:20:40 2022] POST /identity/v3/projects => generated 355 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:20:41.086575 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df05ee28-fb0d-4f7a-bf5e-0e6f99af92c1 admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:41.173395 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df05ee28-fb0d-4f7a-bf5e-0e6f99af92c1 admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:41.178179 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ea1694-2d3c-4004-8c00-a76724661341 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:41.178791 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ea1694-2d3c-4004-8c00-a76724661341 admin admin] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:41.179070 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ea1694-2d3c-4004-8c00-a76724661341 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:41.179344 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ea1694-2d3c-4004-8c00-a76724661341 admin admin] PATH_INFO: `/v3/users` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:41.180071 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ea1694-2d3c-4004-8c00-a76724661341 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:41.181989 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ea1694-2d3c-4004-8c00-a76724661341 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:41.344470 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-575e4bb1-554f-48ba-bfe1-e3914fd6d2ea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5edaf3c7bbb8440dbe132d9909bf9d65'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:41.357350 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-575e4bb1-554f-48ba-bfe1-e3914fd6d2ea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5edaf3c7bbb8440dbe132d9909bf9d65'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:41.364377 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-575e4bb1-554f-48ba-bfe1-e3914fd6d2ea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5edaf3c7bbb8440dbe132d9909bf9d65'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:41.381302 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 869/1780] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Sep 23 15:20:41 2022] POST /identity/v3/users => generated 278 bytes in 295 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 23 15:20:41.385763 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ea1694-2d3c-4004-8c00-a76724661341 admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:41.396196 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ea1694-2d3c-4004-8c00-a76724661341 admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:41.400918 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe20cb9-98bb-4537-a79d-9f72297f30ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:41.406052 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe20cb9-98bb-4537-a79d-9f72297f30ee admin admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:41.406052 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe20cb9-98bb-4537-a79d-9f72297f30ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:41.406052 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe20cb9-98bb-4537-a79d-9f72297f30ee admin admin] PATH_INFO: `/v3/roles` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:41.406052 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe20cb9-98bb-4537-a79d-9f72297f30ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:41.406052 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe20cb9-98bb-4537-a79d-9f72297f30ee admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:41.422211 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 870/1781] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Sep 23 15:20:41 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 23 15:20:41.423485 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe20cb9-98bb-4537-a79d-9f72297f30ee admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:41.433678 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe20cb9-98bb-4537-a79d-9f72297f30ee admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:41.438743 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6833224b-57a7-4705-9fd7-c5a553d58ad8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:41.439354 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6833224b-57a7-4705-9fd7-c5a553d58ad8 admin admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:41.439634 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6833224b-57a7-4705-9fd7-c5a553d58ad8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:41.439910 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6833224b-57a7-4705-9fd7-c5a553d58ad8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:41.440583 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6833224b-57a7-4705-9fd7-c5a553d58ad8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:41.442258 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6833224b-57a7-4705-9fd7-c5a553d58ad8 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:41.453315 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 871/1782] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Sep 23 15:20:41 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 23 15:20:41.457539 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6833224b-57a7-4705-9fd7-c5a553d58ad8 admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:41.468713 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6833224b-57a7-4705-9fd7-c5a553d58ad8 admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:41.469426 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-575e4bb1-554f-48ba-bfe1-e3914fd6d2ea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5edaf3c7bbb8440dbe132d9909bf9d65'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:41.470254 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-575e4bb1-554f-48ba-bfe1-e3914fd6d2ea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5edaf3c7bbb8440dbe132d9909bf9d65'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:41.474055 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af931e8b-909c-4a3d-b291-7fa823228726 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:41.474788 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af931e8b-909c-4a3d-b291-7fa823228726 admin admin] REQUEST_METHOD: `PUT` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:41.475248 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af931e8b-909c-4a3d-b291-7fa823228726 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:41.475609 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af931e8b-909c-4a3d-b291-7fa823228726 admin admin] PATH_INFO: `/v3/projects/24c3c30a9f2945dc80221892debb75d6/users/3ec3bec059b84e52b000ac20f9ef08a6/roles/af257be2dee54d6b942f58650196582c` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:41.477736 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 912/1783] 10.222.0.55 () {58 vars in 1224 bytes} [Fri Sep 23 15:20:40 2022] DELETE /identity/v3/users/5edaf3c7bbb8440dbe132d9909bf9d65 => generated 0 bytes in 523 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:20:41.487935 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedd8991-a6cf-44b9-9d65-bef23ffd1e0f None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:41.488254 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedd8991-a6cf-44b9-9d65-bef23ffd1e0f None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:41.488537 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedd8991-a6cf-44b9-9d65-bef23ffd1e0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:41.493846 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af931e8b-909c-4a3d-b291-7fa823228726 admin admin] RBAC: Authorizing `identity:create_grant(project_id=24c3c30a9f2945dc80221892debb75d6, user_id=3ec3bec059b84e52b000ac20f9ef08a6, role_id=af257be2dee54d6b942f58650196582c)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:41.497001 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af931e8b-909c-4a3d-b291-7fa823228726 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:41.509409 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-fedd8991-a6cf-44b9-9d65-bef23ffd1e0f None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:20:41.510633 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-fedd8991-a6cf-44b9-9d65-bef23ffd1e0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:20:41.583374 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 913/1784] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:20:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 15:20:41.593960 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedd8991-a6cf-44b9-9d65-bef23ffd1e0f None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:41.595444 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-fedd8991-a6cf-44b9-9d65-bef23ffd1e0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:20:41.638784 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 872/1785] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Sep 23 15:20:41 2022] PUT /identity/v3/projects/24c3c30a9f2945dc80221892debb75d6/users/3ec3bec059b84e52b000ac20f9ef08a6/roles/af257be2dee54d6b942f58650196582c => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 15:20:41.650177 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056c7a47-313f-4ce5-890d-41a64840eea8 None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:41.650747 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056c7a47-313f-4ce5-890d-41a64840eea8 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:41.651133 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056c7a47-313f-4ce5-890d-41a64840eea8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:41.653083 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:20:41.666743 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedd8991-a6cf-44b9-9d65-bef23ffd1e0f None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:41.683864 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-056c7a47-313f-4ce5-890d-41a64840eea8 None None] MFA Rules not processed for user `3ec3bec059b84e52b000ac20f9ef08a6`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:20:41.685315 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-056c7a47-313f-4ce5-890d-41a64840eea8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:20:41.720969 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc933a5-c7b2-4299-87ac-b2dadccd3319 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd259178570864ffeb936ee10dfec6b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:41.721546 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc933a5-c7b2-4299-87ac-b2dadccd3319 service neutron] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:41.721640 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc933a5-c7b2-4299-87ac-b2dadccd3319 service neutron] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:41.721890 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc933a5-c7b2-4299-87ac-b2dadccd3319 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:41.759089 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 873/1786] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:20:41 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 15:20:41.764236 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056c7a47-313f-4ce5-890d-41a64840eea8 None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:41.766153 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-056c7a47-313f-4ce5-890d-41a64840eea8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:20:41.790353 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc933a5-c7b2-4299-87ac-b2dadccd3319 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:41.790353 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc933a5-c7b2-4299-87ac-b2dadccd3319 service neutron] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:41.798886 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 914/1787] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:20:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 23 15:20:41.858940 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056c7a47-313f-4ce5-890d-41a64840eea8 None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:41.858940 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d2a466-15ca-42d1-b3ad-0ba1dc30f24b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:41.858940 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d2a466-15ca-42d1-b3ad-0ba1dc30f24b admin admin] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:41.858940 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d2a466-15ca-42d1-b3ad-0ba1dc30f24b admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:41.859327 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d2a466-15ca-42d1-b3ad-0ba1dc30f24b admin admin] PATH_INFO: `/v3/projects` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:41.859327 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d2a466-15ca-42d1-b3ad-0ba1dc30f24b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:41.859327 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d2a466-15ca-42d1-b3ad-0ba1dc30f24b admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:42.094487 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 874/1788] 10.222.0.55 () {60 vars in 1153 bytes} [Fri Sep 23 15:20:41 2022] POST /identity/v3/projects => generated 355 bytes in 330 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 23 15:20:42.098795 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc933a5-c7b2-4299-87ac-b2dadccd3319 service neutron] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:42.116940 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc933a5-c7b2-4299-87ac-b2dadccd3319 service neutron] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:42.300645 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2d90cf-d813-4a01-aef3-451a35f4a4d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:42.301301 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2d90cf-d813-4a01-aef3-451a35f4a4d1 admin admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:42.301586 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2d90cf-d813-4a01-aef3-451a35f4a4d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:42.301844 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2d90cf-d813-4a01-aef3-451a35f4a4d1 admin admin] PATH_INFO: `/v3/users` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:42.302602 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2d90cf-d813-4a01-aef3-451a35f4a4d1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:42.308200 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2d90cf-d813-4a01-aef3-451a35f4a4d1 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:42.591111 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 915/1789] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Sep 23 15:20:42 2022] POST /identity/v3/users => generated 284 bytes in 494 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:20:42.606023 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d2a466-15ca-42d1-b3ad-0ba1dc30f24b admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:42.615622 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d2a466-15ca-42d1-b3ad-0ba1dc30f24b admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:42.622544 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb287c3-2087-45be-9c8b-4eaa36f3c73f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:42.623481 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb287c3-2087-45be-9c8b-4eaa36f3c73f admin admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:42.623933 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb287c3-2087-45be-9c8b-4eaa36f3c73f admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:42.624455 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb287c3-2087-45be-9c8b-4eaa36f3c73f admin admin] PATH_INFO: `/v3/roles` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:42.625493 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb287c3-2087-45be-9c8b-4eaa36f3c73f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:42.628279 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb287c3-2087-45be-9c8b-4eaa36f3c73f admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:42.644041 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 875/1790] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Sep 23 15:20:42 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:20:42.655704 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2d90cf-d813-4a01-aef3-451a35f4a4d1 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:42.657090 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb287c3-2087-45be-9c8b-4eaa36f3c73f admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:42.659187 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-cdb287c3-2087-45be-9c8b-4eaa36f3c73f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:20:42.666462 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2d90cf-d813-4a01-aef3-451a35f4a4d1 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:42.672535 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174bce06-4952-4aab-a0b0-3828a1fcaa79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:42.673198 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174bce06-4952-4aab-a0b0-3828a1fcaa79 admin admin] REQUEST_METHOD: `PUT` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:42.673482 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174bce06-4952-4aab-a0b0-3828a1fcaa79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:42.673814 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174bce06-4952-4aab-a0b0-3828a1fcaa79 admin admin] PATH_INFO: `/v3/projects/0cb6baa322f24f6bb24f38a4b921052e/users/8f596be90fd4448cb93eb90578d1563d/roles/8e2919e79df040e8926f21607818b58c` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:42.692082 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb287c3-2087-45be-9c8b-4eaa36f3c73f admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:42.700456 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d94862-d535-470b-9af1-af37ebeff60b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:42.701106 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d94862-d535-470b-9af1-af37ebeff60b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:42.701383 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d94862-d535-470b-9af1-af37ebeff60b admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:42.701657 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d94862-d535-470b-9af1-af37ebeff60b admin admin] PATH_INFO: `/v3/projects/8a7ce67550004fb9baa3fd6eb2d52f32` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:42.702000 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174bce06-4952-4aab-a0b0-3828a1fcaa79 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0cb6baa322f24f6bb24f38a4b921052e, user_id=8f596be90fd4448cb93eb90578d1563d, role_id=8e2919e79df040e8926f21607818b58c)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:42.702786 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d94862-d535-470b-9af1-af37ebeff60b admin admin] RBAC: Authorizing `identity:delete_project(project_id=8a7ce67550004fb9baa3fd6eb2d52f32)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:42.704852 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d94862-d535-470b-9af1-af37ebeff60b admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:42.713171 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174bce06-4952-4aab-a0b0-3828a1fcaa79 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:42.842198 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 916/1791] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Sep 23 15:20:42 2022] PUT /identity/v3/projects/0cb6baa322f24f6bb24f38a4b921052e/users/8f596be90fd4448cb93eb90578d1563d/roles/8e2919e79df040e8926f21607818b58c => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:20:42.846752 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174bce06-4952-4aab-a0b0-3828a1fcaa79 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:42.857481 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-12d94862-d535-470b-9af1-af37ebeff60b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8a7ce67550004fb9baa3fd6eb2d52f32', 'request_id': 'req-12d94862-d535-470b-9af1-af37ebeff60b'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:42.863418 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-12d94862-d535-470b-9af1-af37ebeff60b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8a7ce67550004fb9baa3fd6eb2d52f32', 'request_id': 'req-12d94862-d535-470b-9af1-af37ebeff60b'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:42.921577 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174bce06-4952-4aab-a0b0-3828a1fcaa79 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:42.980495 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e53fa96-45e6-41ec-9c12-26fa510ae1ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:42.981091 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e53fa96-45e6-41ec-9c12-26fa510ae1ad admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:42.981368 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e53fa96-45e6-41ec-9c12-26fa510ae1ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:42.981729 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e53fa96-45e6-41ec-9c12-26fa510ae1ad admin admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:42.982365 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e53fa96-45e6-41ec-9c12-26fa510ae1ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:42.983950 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e53fa96-45e6-41ec-9c12-26fa510ae1ad admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:42.993532 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 917/1792] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Sep 23 15:20:42 2022] GET /identity/v3/roles => generated 1420 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:20:42.998303 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e53fa96-45e6-41ec-9c12-26fa510ae1ad admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:43.007732 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e53fa96-45e6-41ec-9c12-26fa510ae1ad admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:43.013652 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05462d4d-c933-458d-b56e-0dba84f118a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:43.014251 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05462d4d-c933-458d-b56e-0dba84f118a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:43.014537 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05462d4d-c933-458d-b56e-0dba84f118a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:43.015074 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05462d4d-c933-458d-b56e-0dba84f118a1 admin admin] PATH_INFO: `/v3/projects/0cb6baa322f24f6bb24f38a4b921052e/users/8f596be90fd4448cb93eb90578d1563d/roles/8e2919e79df040e8926f21607818b58c` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:43.017021 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05462d4d-c933-458d-b56e-0dba84f118a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0cb6baa322f24f6bb24f38a4b921052e, user_id=8f596be90fd4448cb93eb90578d1563d, role_id=8e2919e79df040e8926f21607818b58c)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:43.019666 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05462d4d-c933-458d-b56e-0dba84f118a1 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:43.031428 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 918/1793] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Sep 23 15:20:42 2022] PUT /identity/v3/projects/0cb6baa322f24f6bb24f38a4b921052e/users/8f596be90fd4448cb93eb90578d1563d/roles/8e2919e79df040e8926f21607818b58c => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 15:20:43.041442 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f018c560-cb55-44f3-9b65-fe1f8c8de063 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:43.041769 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f018c560-cb55-44f3-9b65-fe1f8c8de063 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:43.042564 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f018c560-cb55-44f3-9b65-fe1f8c8de063 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:43.051938 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-12d94862-d535-470b-9af1-af37ebeff60b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8a7ce67550004fb9baa3fd6eb2d52f32', 'request_id': 'req-12d94862-d535-470b-9af1-af37ebeff60b'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:43.065588 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-12d94862-d535-470b-9af1-af37ebeff60b admin admin] The token cache is being invalidate because project 8a7ce67550004fb9baa3fd6eb2d52f32 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81993) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 15:20:43.066084 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-12d94862-d535-470b-9af1-af37ebeff60b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:43.066886 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-12d94862-d535-470b-9af1-af37ebeff60b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:43.068471 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 876/1794] 10.222.0.55 () {58 vars in 1233 bytes} [Fri Sep 23 15:20:42 2022] DELETE /identity/v3/projects/8a7ce67550004fb9baa3fd6eb2d52f32 => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:20:43.072507 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-f018c560-cb55-44f3-9b65-fe1f8c8de063 None None] MFA Rules not processed for user `8f596be90fd4448cb93eb90578d1563d`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:20:43.075963 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-f018c560-cb55-44f3-9b65-fe1f8c8de063 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:20:43.078397 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d94862-d535-470b-9af1-af37ebeff60b admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:43.080192 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-12d94862-d535-470b-9af1-af37ebeff60b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:20:43.093997 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d94862-d535-470b-9af1-af37ebeff60b admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:43.145360 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 919/1795] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:20:43 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 15:20:43.156508 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdef39e2-af55-4375-ba3b-3a1d7bbc1ebe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:43.156508 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdef39e2-af55-4375-ba3b-3a1d7bbc1ebe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:43.156508 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdef39e2-af55-4375-ba3b-3a1d7bbc1ebe admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:43.156508 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdef39e2-af55-4375-ba3b-3a1d7bbc1ebe admin admin] PATH_INFO: `/v3/users/3167e37d05d74802a3215b177b42226e` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:43.157092 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdef39e2-af55-4375-ba3b-3a1d7bbc1ebe admin admin] RBAC: Authorizing `identity:delete_user(user_id=3167e37d05d74802a3215b177b42226e)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:43.157847 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f018c560-cb55-44f3-9b65-fe1f8c8de063 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:43.159094 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdef39e2-af55-4375-ba3b-3a1d7bbc1ebe admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:43.159204 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-f018c560-cb55-44f3-9b65-fe1f8c8de063 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:20:43.216699 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f018c560-cb55-44f3-9b65-fe1f8c8de063 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:43.220273 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be3ea29-440a-43d9-8f99-1d47bea3248d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feca6278be904ef8b0dc580a67e477f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:43.220884 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be3ea29-440a-43d9-8f99-1d47bea3248d service cinder] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:43.221169 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be3ea29-440a-43d9-8f99-1d47bea3248d service cinder] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:43.221446 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be3ea29-440a-43d9-8f99-1d47bea3248d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:43.231042 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be3ea29-440a-43d9-8f99-1d47bea3248d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:43.232659 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be3ea29-440a-43d9-8f99-1d47bea3248d service cinder] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:43.242173 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 920/1796] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:20:43 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:20:43.503102 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-cdef39e2-af55-4375-ba3b-3a1d7bbc1ebe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3167e37d05d74802a3215b177b42226e'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:43.516460 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-cdef39e2-af55-4375-ba3b-3a1d7bbc1ebe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3167e37d05d74802a3215b177b42226e'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:43.524089 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-cdef39e2-af55-4375-ba3b-3a1d7bbc1ebe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3167e37d05d74802a3215b177b42226e'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:43.534785 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be3ea29-440a-43d9-8f99-1d47bea3248d service cinder] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:43.543966 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be3ea29-440a-43d9-8f99-1d47bea3248d service cinder] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:43.547606 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a53bf1-ef1b-46cb-9328-8fc5ff39e0d2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feca6278be904ef8b0dc580a67e477f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:43.548245 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a53bf1-ef1b-46cb-9328-8fc5ff39e0d2 service cinder] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:43.550835 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a53bf1-ef1b-46cb-9328-8fc5ff39e0d2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:43.550835 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a53bf1-ef1b-46cb-9328-8fc5ff39e0d2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:43.550835 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-38a53bf1-ef1b-46cb-9328-8fc5ff39e0d2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:20:43.611349 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a53bf1-ef1b-46cb-9328-8fc5ff39e0d2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:43.612816 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a53bf1-ef1b-46cb-9328-8fc5ff39e0d2 service cinder] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:43.622639 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 921/1797] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:20:43 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 23 15:20:43.632362 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-cdef39e2-af55-4375-ba3b-3a1d7bbc1ebe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3167e37d05d74802a3215b177b42226e'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:43.633110 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-cdef39e2-af55-4375-ba3b-3a1d7bbc1ebe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3167e37d05d74802a3215b177b42226e'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:43.641363 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 877/1798] 10.222.0.55 () {58 vars in 1224 bytes} [Fri Sep 23 15:20:43 2022] DELETE /identity/v3/users/3167e37d05d74802a3215b177b42226e => generated 0 bytes in 563 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 15:20:44.275318 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a53bf1-ef1b-46cb-9328-8fc5ff39e0d2 service cinder] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:44.293114 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a53bf1-ef1b-46cb-9328-8fc5ff39e0d2 service cinder] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:44.361857 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44991ea4-4fd5-42ec-a4fc-f30629009d1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:44.366629 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44991ea4-4fd5-42ec-a4fc-f30629009d1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:44.366629 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44991ea4-4fd5-42ec-a4fc-f30629009d1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:44.366629 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44991ea4-4fd5-42ec-a4fc-f30629009d1a admin admin] PATH_INFO: `/v3/projects/885bc6c7a0914f41b8c8a078748c64de` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:44.366629 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44991ea4-4fd5-42ec-a4fc-f30629009d1a admin admin] RBAC: Authorizing `identity:delete_project(project_id=885bc6c7a0914f41b8c8a078748c64de)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:44.366629 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44991ea4-4fd5-42ec-a4fc-f30629009d1a admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:44.477967 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-44991ea4-4fd5-42ec-a4fc-f30629009d1a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '885bc6c7a0914f41b8c8a078748c64de', 'request_id': 'req-44991ea4-4fd5-42ec-a4fc-f30629009d1a'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:44.484182 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-44991ea4-4fd5-42ec-a4fc-f30629009d1a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '885bc6c7a0914f41b8c8a078748c64de', 'request_id': 'req-44991ea4-4fd5-42ec-a4fc-f30629009d1a'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:44.489666 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-44991ea4-4fd5-42ec-a4fc-f30629009d1a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '885bc6c7a0914f41b8c8a078748c64de', 'request_id': 'req-44991ea4-4fd5-42ec-a4fc-f30629009d1a'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:44.607310 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-44991ea4-4fd5-42ec-a4fc-f30629009d1a admin admin] The token cache is being invalidate because project 885bc6c7a0914f41b8c8a078748c64de was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81994) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 15:20:44.607690 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-44991ea4-4fd5-42ec-a4fc-f30629009d1a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:44.608360 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-44991ea4-4fd5-42ec-a4fc-f30629009d1a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:44.611504 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 922/1799] 10.222.0.55 () {58 vars in 1233 bytes} [Fri Sep 23 15:20:44 2022] DELETE /identity/v3/projects/885bc6c7a0914f41b8c8a078748c64de => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 15:20:44.614998 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dd850b-3aed-42a1-8503-a3cb9c90c025 None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:44.615290 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dd850b-3aed-42a1-8503-a3cb9c90c025 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:44.615571 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dd850b-3aed-42a1-8503-a3cb9c90c025 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:44.636657 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-38dd850b-3aed-42a1-8503-a3cb9c90c025 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:20:44.637827 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-38dd850b-3aed-42a1-8503-a3cb9c90c025 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:20:44.703506 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 878/1800] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:20:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:20:44.712637 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8435c62f-ffa8-48c8-86b8-041ecc09715b None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:44.712926 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8435c62f-ffa8-48c8-86b8-041ecc09715b None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:44.713254 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8435c62f-ffa8-48c8-86b8-041ecc09715b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:44.734345 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-8435c62f-ffa8-48c8-86b8-041ecc09715b None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:20:44.735653 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-8435c62f-ffa8-48c8-86b8-041ecc09715b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:20:44.743903 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 923/1801] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:20:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:20:44.749148 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38dd850b-3aed-42a1-8503-a3cb9c90c025 None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:44.758930 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38dd850b-3aed-42a1-8503-a3cb9c90c025 None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:44.764154 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894ccd0a-4b49-400c-87cf-2a9d47606ca3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:44.767460 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894ccd0a-4b49-400c-87cf-2a9d47606ca3 admin admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:44.767460 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894ccd0a-4b49-400c-87cf-2a9d47606ca3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:44.767460 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894ccd0a-4b49-400c-87cf-2a9d47606ca3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:44.767460 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894ccd0a-4b49-400c-87cf-2a9d47606ca3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:20:44.767460 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894ccd0a-4b49-400c-87cf-2a9d47606ca3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:44.768111 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894ccd0a-4b49-400c-87cf-2a9d47606ca3 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:44.780345 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 879/1802] 10.222.0.55 () {58 vars in 1153 bytes} [Fri Sep 23 15:20:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:20:44.784659 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8435c62f-ffa8-48c8-86b8-041ecc09715b None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:44.803325 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8435c62f-ffa8-48c8-86b8-041ecc09715b None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:44.803325 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c657c92-1a21-42e7-a756-343cd3887680 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:44.803325 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c657c92-1a21-42e7-a756-343cd3887680 admin admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:44.803325 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c657c92-1a21-42e7-a756-343cd3887680 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:44.803623 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c657c92-1a21-42e7-a756-343cd3887680 admin admin] PATH_INFO: `/v3/projects` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:44.803623 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c657c92-1a21-42e7-a756-343cd3887680 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:44.804189 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c657c92-1a21-42e7-a756-343cd3887680 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:44.894923 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:20:44.937992 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 924/1803] 10.222.0.55 () {60 vars in 1153 bytes} [Fri Sep 23 15:20:44 2022] POST /identity/v3/projects => generated 379 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:20:44.942392 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894ccd0a-4b49-400c-87cf-2a9d47606ca3 admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:44.953027 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894ccd0a-4b49-400c-87cf-2a9d47606ca3 admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:45.013244 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d60adb8-e8cc-49b7-9375-2726d2072ed2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:45.014020 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d60adb8-e8cc-49b7-9375-2726d2072ed2 admin admin] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:45.014324 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d60adb8-e8cc-49b7-9375-2726d2072ed2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:45.014867 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d60adb8-e8cc-49b7-9375-2726d2072ed2 admin admin] PATH_INFO: `/v3/users` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:45.015749 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d60adb8-e8cc-49b7-9375-2726d2072ed2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:45.017726 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d60adb8-e8cc-49b7-9375-2726d2072ed2 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:45.150101 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 880/1804] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Sep 23 15:20:44 2022] POST /identity/v3/users => generated 290 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:20:45.155502 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c657c92-1a21-42e7-a756-343cd3887680 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:45.172811 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c657c92-1a21-42e7-a756-343cd3887680 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:45.193807 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61dec9bd-f9f1-4005-8b25-18abd6c95fb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:45.196798 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dec9bd-f9f1-4005-8b25-18abd6c95fb3 admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:45.196798 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dec9bd-f9f1-4005-8b25-18abd6c95fb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:45.196798 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dec9bd-f9f1-4005-8b25-18abd6c95fb3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:45.199279 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61dec9bd-f9f1-4005-8b25-18abd6c95fb3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:45.203119 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61dec9bd-f9f1-4005-8b25-18abd6c95fb3 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:45.224650 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 925/1805] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Sep 23 15:20:45 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:20:45.228846 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d60adb8-e8cc-49b7-9375-2726d2072ed2 admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:45.244332 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d60adb8-e8cc-49b7-9375-2726d2072ed2 admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:45.253146 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46f9dea-eef6-4d34-9054-6b7376a53c1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:45.253812 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46f9dea-eef6-4d34-9054-6b7376a53c1f admin admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:45.254095 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46f9dea-eef6-4d34-9054-6b7376a53c1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:45.254399 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46f9dea-eef6-4d34-9054-6b7376a53c1f admin admin] PATH_INFO: `/v3/roles` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:45.257146 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46f9dea-eef6-4d34-9054-6b7376a53c1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:45.259361 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46f9dea-eef6-4d34-9054-6b7376a53c1f admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:45.281066 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 881/1806] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Sep 23 15:20:45 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:20:45.282001 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61dec9bd-f9f1-4005-8b25-18abd6c95fb3 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:45.294420 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61dec9bd-f9f1-4005-8b25-18abd6c95fb3 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:45.299836 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882e7be9-17ef-410e-b34b-295d40340b61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:45.300461 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882e7be9-17ef-410e-b34b-295d40340b61 admin admin] REQUEST_METHOD: `PUT` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:45.300827 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882e7be9-17ef-410e-b34b-295d40340b61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:45.301199 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882e7be9-17ef-410e-b34b-295d40340b61 admin admin] PATH_INFO: `/v3/projects/1ae431f789f648029869dc94e5943414/users/51fa3b654a824c4a850692437278bd54/roles/af257be2dee54d6b942f58650196582c` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:45.318047 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882e7be9-17ef-410e-b34b-295d40340b61 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ae431f789f648029869dc94e5943414, user_id=51fa3b654a824c4a850692437278bd54, role_id=af257be2dee54d6b942f58650196582c)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:45.321330 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882e7be9-17ef-410e-b34b-295d40340b61 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:45.462749 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 926/1807] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Sep 23 15:20:45 2022] PUT /identity/v3/projects/1ae431f789f648029869dc94e5943414/users/51fa3b654a824c4a850692437278bd54/roles/af257be2dee54d6b942f58650196582c => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 15:20:45.465289 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fce90dd-1f5a-4c11-9dca-7f353c4fe7ba None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:45.465610 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fce90dd-1f5a-4c11-9dca-7f353c4fe7ba None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:45.465887 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fce90dd-1f5a-4c11-9dca-7f353c4fe7ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:45.497384 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-1fce90dd-1f5a-4c11-9dca-7f353c4fe7ba None None] MFA Rules not processed for user `51fa3b654a824c4a850692437278bd54`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:20:45.498672 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-1fce90dd-1f5a-4c11-9dca-7f353c4fe7ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:20:45.575226 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 882/1808] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:20:45 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 15:20:45.576862 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882e7be9-17ef-410e-b34b-295d40340b61 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:45.643046 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882e7be9-17ef-410e-b34b-295d40340b61 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:45.647570 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58ad7df-2a9e-4012-be80-ef63feeaf132 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:45.648225 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58ad7df-2a9e-4012-be80-ef63feeaf132 admin admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:45.648512 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58ad7df-2a9e-4012-be80-ef63feeaf132 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:45.649025 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58ad7df-2a9e-4012-be80-ef63feeaf132 admin admin] PATH_INFO: `/v3/projects` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:45.649454 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58ad7df-2a9e-4012-be80-ef63feeaf132 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:45.651935 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58ad7df-2a9e-4012-be80-ef63feeaf132 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:45.774661 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 927/1809] 10.222.0.55 () {60 vars in 1153 bytes} [Fri Sep 23 15:20:45 2022] POST /identity/v3/projects => generated 377 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:20:45.777937 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fce90dd-1f5a-4c11-9dca-7f353c4fe7ba None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:45.789244 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fce90dd-1f5a-4c11-9dca-7f353c4fe7ba None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:45.852315 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef97054b-b531-49cd-a624-900bdfcbd36a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:45.852939 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef97054b-b531-49cd-a624-900bdfcbd36a admin admin] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:45.853218 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef97054b-b531-49cd-a624-900bdfcbd36a admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:45.853497 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef97054b-b531-49cd-a624-900bdfcbd36a admin admin] PATH_INFO: `/v3/users` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:45.854126 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef97054b-b531-49cd-a624-900bdfcbd36a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:45.856376 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef97054b-b531-49cd-a624-900bdfcbd36a admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:46.058794 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 883/1810] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Sep 23 15:20:45 2022] POST /identity/v3/users => generated 289 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:20:46.063587 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58ad7df-2a9e-4012-be80-ef63feeaf132 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:46.074425 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58ad7df-2a9e-4012-be80-ef63feeaf132 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:46.079194 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb50fcf-fff6-419c-8257-66b8f711436f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:46.080459 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb50fcf-fff6-419c-8257-66b8f711436f admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:46.080459 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb50fcf-fff6-419c-8257-66b8f711436f admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:46.080678 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb50fcf-fff6-419c-8257-66b8f711436f admin admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:46.081304 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb50fcf-fff6-419c-8257-66b8f711436f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:46.083022 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb50fcf-fff6-419c-8257-66b8f711436f admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:46.093665 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 928/1811] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Sep 23 15:20:46 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:20:46.108630 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef97054b-b531-49cd-a624-900bdfcbd36a admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:46.119180 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef97054b-b531-49cd-a624-900bdfcbd36a admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:46.132692 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b5e1d1-b994-4757-81ff-1715b85ab968 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:46.133331 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b5e1d1-b994-4757-81ff-1715b85ab968 admin admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:46.133622 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b5e1d1-b994-4757-81ff-1715b85ab968 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:46.133892 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b5e1d1-b994-4757-81ff-1715b85ab968 admin admin] PATH_INFO: `/v3/roles` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:46.134549 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b5e1d1-b994-4757-81ff-1715b85ab968 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:46.136946 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b5e1d1-b994-4757-81ff-1715b85ab968 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:46.165521 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 884/1812] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Sep 23 15:20:46 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:20:46.178351 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb50fcf-fff6-419c-8257-66b8f711436f admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:46.183705 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb50fcf-fff6-419c-8257-66b8f711436f admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:46.188505 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f9d999-3437-4349-ac39-453e2592e46a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:46.189166 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f9d999-3437-4349-ac39-453e2592e46a admin admin] REQUEST_METHOD: `PUT` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:46.189452 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f9d999-3437-4349-ac39-453e2592e46a admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:46.189731 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f9d999-3437-4349-ac39-453e2592e46a admin admin] PATH_INFO: `/v3/projects/179b32e84ed64ee49b7a1ccfeca3e83e/users/fd5101f95fad484a92ab9e74c0d0337d/roles/af257be2dee54d6b942f58650196582c` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:46.205607 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f9d999-3437-4349-ac39-453e2592e46a admin admin] RBAC: Authorizing `identity:create_grant(project_id=179b32e84ed64ee49b7a1ccfeca3e83e, user_id=fd5101f95fad484a92ab9e74c0d0337d, role_id=af257be2dee54d6b942f58650196582c)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:46.208307 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f9d999-3437-4349-ac39-453e2592e46a admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:46.315152 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 929/1813] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Sep 23 15:20:46 2022] PUT /identity/v3/projects/179b32e84ed64ee49b7a1ccfeca3e83e/users/fd5101f95fad484a92ab9e74c0d0337d/roles/af257be2dee54d6b942f58650196582c => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:20:46.325741 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7b7b26-f80b-4d43-a409-23649b4c2a99 None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:46.326028 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7b7b26-f80b-4d43-a409-23649b4c2a99 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:46.326298 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7b7b26-f80b-4d43-a409-23649b4c2a99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:46.362631 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-ae7b7b26-f80b-4d43-a409-23649b4c2a99 None None] MFA Rules not processed for user `fd5101f95fad484a92ab9e74c0d0337d`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:20:46.363866 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-ae7b7b26-f80b-4d43-a409-23649b4c2a99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:20:46.435463 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 885/1814] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:20:46 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:20:46.440117 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f9d999-3437-4349-ac39-453e2592e46a admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:46.504429 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f9d999-3437-4349-ac39-453e2592e46a admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:46.509113 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da501177-8ca6-4156-bdc9-3c78512e3da1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:46.509720 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da501177-8ca6-4156-bdc9-3c78512e3da1 admin admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:46.510003 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da501177-8ca6-4156-bdc9-3c78512e3da1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:46.510318 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da501177-8ca6-4156-bdc9-3c78512e3da1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:46.511546 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da501177-8ca6-4156-bdc9-3c78512e3da1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:46.513476 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da501177-8ca6-4156-bdc9-3c78512e3da1 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:46.633796 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 930/1815] 10.222.0.55 () {60 vars in 1153 bytes} [Fri Sep 23 15:20:46 2022] POST /identity/v3/projects => generated 379 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:20:46.637823 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7b7b26-f80b-4d43-a409-23649b4c2a99 None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:46.649833 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7b7b26-f80b-4d43-a409-23649b4c2a99 None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:46.711077 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6a32b1-ca38-46ed-8cbf-8cf8186eb3df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:46.711699 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6a32b1-ca38-46ed-8cbf-8cf8186eb3df admin admin] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:46.712009 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6a32b1-ca38-46ed-8cbf-8cf8186eb3df admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:46.712307 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6a32b1-ca38-46ed-8cbf-8cf8186eb3df admin admin] PATH_INFO: `/v3/users` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:46.712941 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6a32b1-ca38-46ed-8cbf-8cf8186eb3df admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:46.715656 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6a32b1-ca38-46ed-8cbf-8cf8186eb3df admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:46.853262 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 886/1816] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Sep 23 15:20:46 2022] POST /identity/v3/users => generated 296 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 23 15:20:46.857327 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da501177-8ca6-4156-bdc9-3c78512e3da1 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:46.867839 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da501177-8ca6-4156-bdc9-3c78512e3da1 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:46.873495 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071976f7-de84-4046-b853-3810097e9679 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:46.874106 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071976f7-de84-4046-b853-3810097e9679 admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:46.874391 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071976f7-de84-4046-b853-3810097e9679 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:46.874665 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071976f7-de84-4046-b853-3810097e9679 admin admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:46.875323 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071976f7-de84-4046-b853-3810097e9679 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:46.876987 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071976f7-de84-4046-b853-3810097e9679 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:46.889002 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 931/1817] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Sep 23 15:20:46 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:20:46.893655 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6a32b1-ca38-46ed-8cbf-8cf8186eb3df admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:46.904851 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6a32b1-ca38-46ed-8cbf-8cf8186eb3df admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:46.914885 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cda14c-8e96-4293-867c-92dfe030dc7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:46.914885 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cda14c-8e96-4293-867c-92dfe030dc7f admin admin] REQUEST_METHOD: `PUT` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:46.914885 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cda14c-8e96-4293-867c-92dfe030dc7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:46.914885 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cda14c-8e96-4293-867c-92dfe030dc7f admin admin] PATH_INFO: `/v3/projects/3d4edfe610d34d88922171446897dfbb/users/54823785411749979d220c33c424c522/roles/8e2919e79df040e8926f21607818b58c` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:46.927471 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08cda14c-8e96-4293-867c-92dfe030dc7f admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d4edfe610d34d88922171446897dfbb, user_id=54823785411749979d220c33c424c522, role_id=8e2919e79df040e8926f21607818b58c)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:46.930194 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08cda14c-8e96-4293-867c-92dfe030dc7f admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:47.090589 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 887/1818] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Sep 23 15:20:46 2022] PUT /identity/v3/projects/3d4edfe610d34d88922171446897dfbb/users/54823785411749979d220c33c424c522/roles/8e2919e79df040e8926f21607818b58c => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:20:47.094475 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071976f7-de84-4046-b853-3810097e9679 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:47.104843 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071976f7-de84-4046-b853-3810097e9679 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:47.167169 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b2b252-149d-4f88-aab2-4abba9b38ddb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:47.168043 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b2b252-149d-4f88-aab2-4abba9b38ddb admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:47.168324 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b2b252-149d-4f88-aab2-4abba9b38ddb admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:47.168595 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b2b252-149d-4f88-aab2-4abba9b38ddb admin admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:47.169245 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b2b252-149d-4f88-aab2-4abba9b38ddb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:47.170973 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b2b252-149d-4f88-aab2-4abba9b38ddb admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:47.181484 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 932/1819] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Sep 23 15:20:47 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:20:47.186530 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cda14c-8e96-4293-867c-92dfe030dc7f admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:47.197358 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cda14c-8e96-4293-867c-92dfe030dc7f admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:47.203081 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89f2159-8f66-479b-be05-aef698799340 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:47.203802 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89f2159-8f66-479b-be05-aef698799340 admin admin] REQUEST_METHOD: `PUT` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:47.204099 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89f2159-8f66-479b-be05-aef698799340 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:47.205364 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89f2159-8f66-479b-be05-aef698799340 admin admin] PATH_INFO: `/v3/projects/3d4edfe610d34d88922171446897dfbb/users/54823785411749979d220c33c424c522/roles/8e2919e79df040e8926f21607818b58c` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:47.206510 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89f2159-8f66-479b-be05-aef698799340 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d4edfe610d34d88922171446897dfbb, user_id=54823785411749979d220c33c424c522, role_id=8e2919e79df040e8926f21607818b58c)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:47.209249 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89f2159-8f66-479b-be05-aef698799340 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:47.218927 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:20:47.221335 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 888/1820] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Sep 23 15:20:47 2022] PUT /identity/v3/projects/3d4edfe610d34d88922171446897dfbb/users/54823785411749979d220c33c424c522/roles/8e2919e79df040e8926f21607818b58c => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:20:47.232776 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ce1a38-a8ad-46ef-97ff-08eeba38c2f9 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:47.233070 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ce1a38-a8ad-46ef-97ff-08eeba38c2f9 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:47.233357 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ce1a38-a8ad-46ef-97ff-08eeba38c2f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:47.265688 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-a8ce1a38-a8ad-46ef-97ff-08eeba38c2f9 None None] MFA Rules not processed for user `54823785411749979d220c33c424c522`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:20:47.267304 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-a8ce1a38-a8ad-46ef-97ff-08eeba38c2f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:20:47.353645 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 933/1821] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:20:47 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 15:20:47.365358 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89f2159-8f66-479b-be05-aef698799340 admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:47.367026 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-a89f2159-8f66-479b-be05-aef698799340 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:20:47.425507 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89f2159-8f66-479b-be05-aef698799340 admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:47.429343 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f59551-1889-4b31-8904-38e6d09f74d3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feca6278be904ef8b0dc580a67e477f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:47.429992 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f59551-1889-4b31-8904-38e6d09f74d3 service cinder] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:47.430270 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f59551-1889-4b31-8904-38e6d09f74d3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:47.430547 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f59551-1889-4b31-8904-38e6d09f74d3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:47.440981 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f59551-1889-4b31-8904-38e6d09f74d3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:47.442465 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f59551-1889-4b31-8904-38e6d09f74d3 service cinder] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:47.453776 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 889/1822] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:20:47 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:20:47.636994 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ce1a38-a8ad-46ef-97ff-08eeba38c2f9 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:47.647106 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ce1a38-a8ad-46ef-97ff-08eeba38c2f9 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:47.650726 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d8f81a-d68e-40e8-9512-f7fa6aa9637b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feca6278be904ef8b0dc580a67e477f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:47.651442 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d8f81a-d68e-40e8-9512-f7fa6aa9637b service cinder] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:47.651720 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d8f81a-d68e-40e8-9512-f7fa6aa9637b service cinder] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:47.652014 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d8f81a-d68e-40e8-9512-f7fa6aa9637b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:47.709044 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d8f81a-d68e-40e8-9512-f7fa6aa9637b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:47.710467 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d8f81a-d68e-40e8-9512-f7fa6aa9637b service cinder] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:47.720472 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 934/1823] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:20:47 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 23 15:20:48.003297 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:20:49.047089 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:20:51.954331 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:20:52.603927 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:20:53.094041 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f59551-1889-4b31-8904-38e6d09f74d3 service cinder] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:53.168784 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f59551-1889-4b31-8904-38e6d09f74d3 service cinder] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:53.173667 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d4213c-8831-4695-ab00-9c4c8e8c6ec6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:53.174380 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d4213c-8831-4695-ab00-9c4c8e8c6ec6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:53.174677 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d4213c-8831-4695-ab00-9c4c8e8c6ec6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:53.174952 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d4213c-8831-4695-ab00-9c4c8e8c6ec6 admin admin] PATH_INFO: `/v3/users/51fa3b654a824c4a850692437278bd54` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:53.176321 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d4213c-8831-4695-ab00-9c4c8e8c6ec6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=51fa3b654a824c4a850692437278bd54)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:53.178452 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d4213c-8831-4695-ab00-9c4c8e8c6ec6 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:53.533136 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-f7d4213c-8831-4695-ab00-9c4c8e8c6ec6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '51fa3b654a824c4a850692437278bd54'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:53.547173 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-f7d4213c-8831-4695-ab00-9c4c8e8c6ec6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '51fa3b654a824c4a850692437278bd54'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:53.554416 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-f7d4213c-8831-4695-ab00-9c4c8e8c6ec6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '51fa3b654a824c4a850692437278bd54'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:53.706142 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-f7d4213c-8831-4695-ab00-9c4c8e8c6ec6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '51fa3b654a824c4a850692437278bd54'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:53.707379 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-f7d4213c-8831-4695-ab00-9c4c8e8c6ec6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '51fa3b654a824c4a850692437278bd54'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:53.716540 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 890/1824] 10.222.0.55 () {58 vars in 1224 bytes} [Fri Sep 23 15:20:53 2022] DELETE /identity/v3/users/51fa3b654a824c4a850692437278bd54 => generated 0 bytes in 623 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:20:53.721754 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f6cb74-c3ac-413e-8e91-3515a541fcd2 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:53.722270 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f6cb74-c3ac-413e-8e91-3515a541fcd2 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:53.722348 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f6cb74-c3ac-413e-8e91-3515a541fcd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:53.750999 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-b8f6cb74-c3ac-413e-8e91-3515a541fcd2 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:20:53.752447 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-b8f6cb74-c3ac-413e-8e91-3515a541fcd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:20:53.840829 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 935/1825] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:20:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:20:53.851287 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d4213c-8831-4695-ab00-9c4c8e8c6ec6 admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:53.853283 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-f7d4213c-8831-4695-ab00-9c4c8e8c6ec6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:20:53.979467 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d4213c-8831-4695-ab00-9c4c8e8c6ec6 admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:53.983115 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6278f13a-58af-45a6-9bb6-5a9cc7b4ec26 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd259178570864ffeb936ee10dfec6b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:53.983783 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6278f13a-58af-45a6-9bb6-5a9cc7b4ec26 service neutron] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:53.984088 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6278f13a-58af-45a6-9bb6-5a9cc7b4ec26 service neutron] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:53.984361 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6278f13a-58af-45a6-9bb6-5a9cc7b4ec26 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:53.995020 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6278f13a-58af-45a6-9bb6-5a9cc7b4ec26 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:53.996574 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6278f13a-58af-45a6-9bb6-5a9cc7b4ec26 service neutron] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:54.009486 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 891/1826] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:20:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:20:54.791971 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f6cb74-c3ac-413e-8e91-3515a541fcd2 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:54.794928 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-b8f6cb74-c3ac-413e-8e91-3515a541fcd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:20:54.810192 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f6cb74-c3ac-413e-8e91-3515a541fcd2 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:54.817098 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc5e977-db5d-46a2-bfb3-551f7af8f06b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:54.817848 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc5e977-db5d-46a2-bfb3-551f7af8f06b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:54.818083 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc5e977-db5d-46a2-bfb3-551f7af8f06b admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:54.818354 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc5e977-db5d-46a2-bfb3-551f7af8f06b admin admin] PATH_INFO: `/v3/projects/1ae431f789f648029869dc94e5943414` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:54.819472 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc5e977-db5d-46a2-bfb3-551f7af8f06b admin admin] RBAC: Authorizing `identity:delete_project(project_id=1ae431f789f648029869dc94e5943414)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:54.821436 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc5e977-db5d-46a2-bfb3-551f7af8f06b admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:54.995126 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-1dc5e977-db5d-46a2-bfb3-551f7af8f06b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1ae431f789f648029869dc94e5943414', 'request_id': 'req-1dc5e977-db5d-46a2-bfb3-551f7af8f06b'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:55.001738 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-1dc5e977-db5d-46a2-bfb3-551f7af8f06b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1ae431f789f648029869dc94e5943414', 'request_id': 'req-1dc5e977-db5d-46a2-bfb3-551f7af8f06b'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:55.007726 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-1dc5e977-db5d-46a2-bfb3-551f7af8f06b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1ae431f789f648029869dc94e5943414', 'request_id': 'req-1dc5e977-db5d-46a2-bfb3-551f7af8f06b'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:55.147627 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-1dc5e977-db5d-46a2-bfb3-551f7af8f06b admin admin] The token cache is being invalidate because project 1ae431f789f648029869dc94e5943414 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81994) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 15:20:55.147921 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-1dc5e977-db5d-46a2-bfb3-551f7af8f06b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:55.148606 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-1dc5e977-db5d-46a2-bfb3-551f7af8f06b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:55.150410 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 936/1827] 10.222.0.55 () {58 vars in 1233 bytes} [Fri Sep 23 15:20:54 2022] DELETE /identity/v3/projects/1ae431f789f648029869dc94e5943414 => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:20:55.154568 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6278f13a-58af-45a6-9bb6-5a9cc7b4ec26 service neutron] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:55.165395 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6278f13a-58af-45a6-9bb6-5a9cc7b4ec26 service neutron] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:55.224796 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccc2f7a-e95f-4cf3-b4a6-d0a0d4f4667f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:55.225438 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccc2f7a-e95f-4cf3-b4a6-d0a0d4f4667f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:55.225721 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccc2f7a-e95f-4cf3-b4a6-d0a0d4f4667f admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:55.226000 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccc2f7a-e95f-4cf3-b4a6-d0a0d4f4667f admin admin] PATH_INFO: `/v3/users/fd5101f95fad484a92ab9e74c0d0337d` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:55.227137 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccc2f7a-e95f-4cf3-b4a6-d0a0d4f4667f admin admin] RBAC: Authorizing `identity:delete_user(user_id=fd5101f95fad484a92ab9e74c0d0337d)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:55.230061 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccc2f7a-e95f-4cf3-b4a6-d0a0d4f4667f admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:55.516419 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-4ccc2f7a-e95f-4cf3-b4a6-d0a0d4f4667f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fd5101f95fad484a92ab9e74c0d0337d'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:55.529868 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-4ccc2f7a-e95f-4cf3-b4a6-d0a0d4f4667f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fd5101f95fad484a92ab9e74c0d0337d'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:55.536877 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-4ccc2f7a-e95f-4cf3-b4a6-d0a0d4f4667f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fd5101f95fad484a92ab9e74c0d0337d'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:55.628195 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-4ccc2f7a-e95f-4cf3-b4a6-d0a0d4f4667f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fd5101f95fad484a92ab9e74c0d0337d'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:55.631447 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-4ccc2f7a-e95f-4cf3-b4a6-d0a0d4f4667f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fd5101f95fad484a92ab9e74c0d0337d'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:55.636189 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 892/1828] 10.222.0.55 () {58 vars in 1224 bytes} [Fri Sep 23 15:20:55 2022] DELETE /identity/v3/users/fd5101f95fad484a92ab9e74c0d0337d => generated 0 bytes in 482 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:20:55.928507 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:20:56.187658 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc5e977-db5d-46a2-bfb3-551f7af8f06b admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:56.190661 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-1dc5e977-db5d-46a2-bfb3-551f7af8f06b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:20:56.204037 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc5e977-db5d-46a2-bfb3-551f7af8f06b admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:56.294861 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e75cec-b59f-4768-a218-a5c99f726128 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:56.295470 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e75cec-b59f-4768-a218-a5c99f726128 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:56.295755 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e75cec-b59f-4768-a218-a5c99f726128 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:56.296521 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e75cec-b59f-4768-a218-a5c99f726128 admin admin] PATH_INFO: `/v3/projects/179b32e84ed64ee49b7a1ccfeca3e83e` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:56.298122 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e75cec-b59f-4768-a218-a5c99f726128 admin admin] RBAC: Authorizing `identity:delete_project(project_id=179b32e84ed64ee49b7a1ccfeca3e83e)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:56.304192 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e75cec-b59f-4768-a218-a5c99f726128 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:56.315931 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:20:56.531410 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-51e75cec-b59f-4768-a218-a5c99f726128 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '179b32e84ed64ee49b7a1ccfeca3e83e', 'request_id': 'req-51e75cec-b59f-4768-a218-a5c99f726128'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:56.540249 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-51e75cec-b59f-4768-a218-a5c99f726128 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '179b32e84ed64ee49b7a1ccfeca3e83e', 'request_id': 'req-51e75cec-b59f-4768-a218-a5c99f726128'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:56.559000 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-51e75cec-b59f-4768-a218-a5c99f726128 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '179b32e84ed64ee49b7a1ccfeca3e83e', 'request_id': 'req-51e75cec-b59f-4768-a218-a5c99f726128'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:56.565103 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53071253-62e3-46e8-9803-b4e09cefbf6e None None] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:56.566471 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53071253-62e3-46e8-9803-b4e09cefbf6e None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:56.566964 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53071253-62e3-46e8-9803-b4e09cefbf6e None None] PATH_INFO: `/` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:56.569649 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 893/1829] 10.222.0.55 () {54 vars in 921 bytes} [Fri Sep 23 15:20:56 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 23 15:20:56.585728 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dac1b77-720c-40eb-9db6-5755487327b6 None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:56.585728 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dac1b77-720c-40eb-9db6-5755487327b6 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:56.585728 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dac1b77-720c-40eb-9db6-5755487327b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:56.607397 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-8dac1b77-720c-40eb-9db6-5755487327b6 None None] MFA Rules not processed for user `b3c6c7a1325a47be924268e1aa1a7a18`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:20:56.608660 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-8dac1b77-720c-40eb-9db6-5755487327b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:20:56.677632 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 894/1830] 10.222.0.55 () {58 vars in 1018 bytes} [Fri Sep 23 15:20:56 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 15:20:56.693446 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dac1b77-720c-40eb-9db6-5755487327b6 None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:56.694872 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-8dac1b77-720c-40eb-9db6-5755487327b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:20:56.709508 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dac1b77-720c-40eb-9db6-5755487327b6 None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:56.716224 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751f417e-0379-4adc-9230-d9b3bfb962c5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3c6c7a1325a47be924268e1aa1a7a18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:56.716442 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751f417e-0379-4adc-9230-d9b3bfb962c5 service nova] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:56.716733 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751f417e-0379-4adc-9230-d9b3bfb962c5 service nova] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:56.717045 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751f417e-0379-4adc-9230-d9b3bfb962c5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:56.726225 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751f417e-0379-4adc-9230-d9b3bfb962c5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:56.727847 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751f417e-0379-4adc-9230-d9b3bfb962c5 service nova] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:56.743294 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-51e75cec-b59f-4768-a218-a5c99f726128 admin admin] The token cache is being invalidate because project 179b32e84ed64ee49b7a1ccfeca3e83e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81994) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 15:20:56.743956 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-51e75cec-b59f-4768-a218-a5c99f726128 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:56.747302 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-51e75cec-b59f-4768-a218-a5c99f726128 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:56.748626 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 937/1831] 10.222.0.55 () {58 vars in 1233 bytes} [Fri Sep 23 15:20:56 2022] DELETE /identity/v3/projects/179b32e84ed64ee49b7a1ccfeca3e83e => generated 0 bytes in 561 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:20:56.754142 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e75cec-b59f-4768-a218-a5c99f726128 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:56.755794 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-51e75cec-b59f-4768-a218-a5c99f726128 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:20:56.766382 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 895/1832] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:20:56 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 73 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 23 15:20:56.767658 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751f417e-0379-4adc-9230-d9b3bfb962c5 service nova] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:56.771688 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-751f417e-0379-4adc-9230-d9b3bfb962c5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:20:56.861160 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e75cec-b59f-4768-a218-a5c99f726128 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:56.875068 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49807e9a-073a-45f3-a1d0-e848446fdbd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:56.875964 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49807e9a-073a-45f3-a1d0-e848446fdbd3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:56.876340 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49807e9a-073a-45f3-a1d0-e848446fdbd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:56.876842 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49807e9a-073a-45f3-a1d0-e848446fdbd3 admin admin] PATH_INFO: `/v3/users/54823785411749979d220c33c424c522` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:56.879024 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49807e9a-073a-45f3-a1d0-e848446fdbd3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=54823785411749979d220c33c424c522)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:56.881223 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49807e9a-073a-45f3-a1d0-e848446fdbd3 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:56.883238 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751f417e-0379-4adc-9230-d9b3bfb962c5 service nova] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:56.895168 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0b456a-8e77-42ed-b866-c1937d2331e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:56.896140 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0b456a-8e77-42ed-b866-c1937d2331e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:56.896662 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0b456a-8e77-42ed-b866-c1937d2331e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:56.896904 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0b456a-8e77-42ed-b866-c1937d2331e6 admin admin] PATH_INFO: `/v3/users/1f8b374d0cc544559ae4458aedb5a412` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:56.898247 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0b456a-8e77-42ed-b866-c1937d2331e6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1f8b374d0cc544559ae4458aedb5a412)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:56.901760 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0b456a-8e77-42ed-b866-c1937d2331e6 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:57.233846 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-49807e9a-073a-45f3-a1d0-e848446fdbd3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '54823785411749979d220c33c424c522'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:57.246683 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-49807e9a-073a-45f3-a1d0-e848446fdbd3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '54823785411749979d220c33c424c522'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:57.254518 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-49807e9a-073a-45f3-a1d0-e848446fdbd3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '54823785411749979d220c33c424c522'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:57.270614 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-6f0b456a-8e77-42ed-b866-c1937d2331e6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1f8b374d0cc544559ae4458aedb5a412'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:57.285412 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-6f0b456a-8e77-42ed-b866-c1937d2331e6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1f8b374d0cc544559ae4458aedb5a412'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:57.293889 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-6f0b456a-8e77-42ed-b866-c1937d2331e6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1f8b374d0cc544559ae4458aedb5a412'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:57.367064 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-49807e9a-073a-45f3-a1d0-e848446fdbd3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '54823785411749979d220c33c424c522'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:57.368461 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-49807e9a-073a-45f3-a1d0-e848446fdbd3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '54823785411749979d220c33c424c522'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:57.382212 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 938/1833] 10.222.0.55 () {58 vars in 1224 bytes} [Fri Sep 23 15:20:56 2022] DELETE /identity/v3/users/54823785411749979d220c33c424c522 => generated 0 bytes in 625 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:20:57.509235 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-6f0b456a-8e77-42ed-b866-c1937d2331e6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1f8b374d0cc544559ae4458aedb5a412'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:57.510184 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-6f0b456a-8e77-42ed-b866-c1937d2331e6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1f8b374d0cc544559ae4458aedb5a412'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:57.520447 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 896/1834] 10.222.0.55 () {58 vars in 1224 bytes} [Fri Sep 23 15:20:56 2022] DELETE /identity/v3/users/1f8b374d0cc544559ae4458aedb5a412 => generated 0 bytes in 753 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:20:57.534667 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18aa8d5d-96fc-4fa3-abff-a96dd66131a9 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:57.534667 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18aa8d5d-96fc-4fa3-abff-a96dd66131a9 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:57.534667 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18aa8d5d-96fc-4fa3-abff-a96dd66131a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:57.586681 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-18aa8d5d-96fc-4fa3-abff-a96dd66131a9 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:20:57.591068 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-18aa8d5d-96fc-4fa3-abff-a96dd66131a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:20:57.713368 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 939/1835] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:20:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:20:57.751335 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0b456a-8e77-42ed-b866-c1937d2331e6 admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:57.761149 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-6f0b456a-8e77-42ed-b866-c1937d2331e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:20:57.890029 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0b456a-8e77-42ed-b866-c1937d2331e6 admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:57.893657 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491c9fbd-ff53-49c5-a021-535416a5028c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd259178570864ffeb936ee10dfec6b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:57.894306 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491c9fbd-ff53-49c5-a021-535416a5028c service neutron] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:57.895088 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491c9fbd-ff53-49c5-a021-535416a5028c service neutron] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:57.895372 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491c9fbd-ff53-49c5-a021-535416a5028c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:57.904950 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491c9fbd-ff53-49c5-a021-535416a5028c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:57.907107 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491c9fbd-ff53-49c5-a021-535416a5028c service neutron] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:57.917555 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 897/1836] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:20:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:20:58.116448 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18aa8d5d-96fc-4fa3-abff-a96dd66131a9 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:58.120802 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-18aa8d5d-96fc-4fa3-abff-a96dd66131a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:20:58.150322 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18aa8d5d-96fc-4fa3-abff-a96dd66131a9 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:58.159516 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b8038f-d794-493a-b06e-bbea6fbacdf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:58.160863 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b8038f-d794-493a-b06e-bbea6fbacdf1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:58.161520 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b8038f-d794-493a-b06e-bbea6fbacdf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:58.162019 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b8038f-d794-493a-b06e-bbea6fbacdf1 admin admin] PATH_INFO: `/v3/projects/3d4edfe610d34d88922171446897dfbb` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:58.166428 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b8038f-d794-493a-b06e-bbea6fbacdf1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3d4edfe610d34d88922171446897dfbb)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:58.184513 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b8038f-d794-493a-b06e-bbea6fbacdf1 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:58.340333 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-21b8038f-d794-493a-b06e-bbea6fbacdf1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3d4edfe610d34d88922171446897dfbb', 'request_id': 'req-21b8038f-d794-493a-b06e-bbea6fbacdf1'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:58.346658 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-21b8038f-d794-493a-b06e-bbea6fbacdf1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3d4edfe610d34d88922171446897dfbb', 'request_id': 'req-21b8038f-d794-493a-b06e-bbea6fbacdf1'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:58.353765 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-21b8038f-d794-493a-b06e-bbea6fbacdf1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3d4edfe610d34d88922171446897dfbb', 'request_id': 'req-21b8038f-d794-493a-b06e-bbea6fbacdf1'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:58.425341 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:20:58.476430 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-21b8038f-d794-493a-b06e-bbea6fbacdf1 admin admin] The token cache is being invalidate because project 3d4edfe610d34d88922171446897dfbb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81994) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 15:20:58.489727 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-21b8038f-d794-493a-b06e-bbea6fbacdf1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:58.489727 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-21b8038f-d794-493a-b06e-bbea6fbacdf1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:58.489727 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 940/1837] 10.222.0.55 () {58 vars in 1233 bytes} [Fri Sep 23 15:20:58 2022] DELETE /identity/v3/projects/3d4edfe610d34d88922171446897dfbb => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:20:58.490040 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b481f97-810d-4fbe-8920-9c798defc245 None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:58.490040 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b481f97-810d-4fbe-8920-9c798defc245 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:58.490040 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b481f97-810d-4fbe-8920-9c798defc245 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:58.505468 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-0b481f97-810d-4fbe-8920-9c798defc245 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:20:58.506669 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-0b481f97-810d-4fbe-8920-9c798defc245 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:20:58.566816 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 898/1838] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:20:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:20:58.575858 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704c54ca-e68d-4d93-8b00-59a1cde51ca7 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:58.576375 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704c54ca-e68d-4d93-8b00-59a1cde51ca7 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:58.576780 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704c54ca-e68d-4d93-8b00-59a1cde51ca7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:58.593822 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b481f97-810d-4fbe-8920-9c798defc245 None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:58.595288 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-0b481f97-810d-4fbe-8920-9c798defc245 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:20:58.607694 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-704c54ca-e68d-4d93-8b00-59a1cde51ca7 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:20:58.609197 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-704c54ca-e68d-4d93-8b00-59a1cde51ca7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:20:58.609774 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b481f97-810d-4fbe-8920-9c798defc245 None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:58.619866 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b314e83-b122-44ef-b1a6-e40b806c842d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:58.620560 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b314e83-b122-44ef-b1a6-e40b806c842d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:58.620840 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b314e83-b122-44ef-b1a6-e40b806c842d admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:58.624041 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b314e83-b122-44ef-b1a6-e40b806c842d admin admin] PATH_INFO: `/v3/projects/70fee69f2c4b48bf80bf7b3667f52f59` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:58.624172 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 941/1839] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:20:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:20:58.624277 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b314e83-b122-44ef-b1a6-e40b806c842d admin admin] RBAC: Authorizing `identity:delete_project(project_id=70fee69f2c4b48bf80bf7b3667f52f59)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:58.624277 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b314e83-b122-44ef-b1a6-e40b806c842d admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:58.625715 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704c54ca-e68d-4d93-8b00-59a1cde51ca7 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:58.645134 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704c54ca-e68d-4d93-8b00-59a1cde51ca7 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:58.658140 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7228a47a-9a75-4a3f-b78a-410db31b65be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:58.658773 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7228a47a-9a75-4a3f-b78a-410db31b65be admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:58.659052 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7228a47a-9a75-4a3f-b78a-410db31b65be admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:58.659319 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7228a47a-9a75-4a3f-b78a-410db31b65be admin admin] PATH_INFO: `/v3/domains` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:58.659969 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7228a47a-9a75-4a3f-b78a-410db31b65be admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:20:58.660299 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7228a47a-9a75-4a3f-b78a-410db31b65be admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:58.661881 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7228a47a-9a75-4a3f-b78a-410db31b65be admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:58.677813 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 942/1840] 10.222.0.55 () {58 vars in 1153 bytes} [Fri Sep 23 15:20:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:20:58.682030 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7228a47a-9a75-4a3f-b78a-410db31b65be admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:58.694486 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7228a47a-9a75-4a3f-b78a-410db31b65be admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:58.701546 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e0c4da-fdde-4845-8d84-8af09781dc1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:58.702585 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e0c4da-fdde-4845-8d84-8af09781dc1d admin admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:58.707457 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e0c4da-fdde-4845-8d84-8af09781dc1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:58.707457 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e0c4da-fdde-4845-8d84-8af09781dc1d admin admin] PATH_INFO: `/v3/projects` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:58.707457 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e0c4da-fdde-4845-8d84-8af09781dc1d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:58.707457 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e0c4da-fdde-4845-8d84-8af09781dc1d admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:58.809371 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-5b314e83-b122-44ef-b1a6-e40b806c842d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '70fee69f2c4b48bf80bf7b3667f52f59', 'request_id': 'req-5b314e83-b122-44ef-b1a6-e40b806c842d'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:58.823424 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-5b314e83-b122-44ef-b1a6-e40b806c842d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '70fee69f2c4b48bf80bf7b3667f52f59', 'request_id': 'req-5b314e83-b122-44ef-b1a6-e40b806c842d'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:58.878089 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 943/1841] 10.222.0.55 () {60 vars in 1153 bytes} [Fri Sep 23 15:20:58 2022] POST /identity/v3/projects => generated 369 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 23 15:20:58.879250 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e0c4da-fdde-4845-8d84-8af09781dc1d admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:59.007618 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e0c4da-fdde-4845-8d84-8af09781dc1d admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:59.014187 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16062662-23ea-46aa-be8b-58d7db7d196c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:59.015022 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16062662-23ea-46aa-be8b-58d7db7d196c admin admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:59.015720 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16062662-23ea-46aa-be8b-58d7db7d196c admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:59.016183 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16062662-23ea-46aa-be8b-58d7db7d196c admin admin] PATH_INFO: `/v3/users` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:59.017170 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16062662-23ea-46aa-be8b-58d7db7d196c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:59.019454 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16062662-23ea-46aa-be8b-58d7db7d196c admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:59.050862 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-5b314e83-b122-44ef-b1a6-e40b806c842d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '70fee69f2c4b48bf80bf7b3667f52f59', 'request_id': 'req-5b314e83-b122-44ef-b1a6-e40b806c842d'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:59.066697 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-5b314e83-b122-44ef-b1a6-e40b806c842d admin admin] The token cache is being invalidate because project 70fee69f2c4b48bf80bf7b3667f52f59 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81993) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 15:20:59.067232 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-5b314e83-b122-44ef-b1a6-e40b806c842d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:59.068163 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-5b314e83-b122-44ef-b1a6-e40b806c842d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:59.070251 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 899/1842] 10.222.0.55 () {58 vars in 1233 bytes} [Fri Sep 23 15:20:58 2022] DELETE /identity/v3/projects/70fee69f2c4b48bf80bf7b3667f52f59 => generated 0 bytes in 477 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 15:20:59.074126 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b314e83-b122-44ef-b1a6-e40b806c842d admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:59.075605 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-5b314e83-b122-44ef-b1a6-e40b806c842d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:20:59.144603 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b314e83-b122-44ef-b1a6-e40b806c842d admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:59.149280 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5860a72e-fc53-45dd-93b6-8b8358c36723 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:59.149969 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5860a72e-fc53-45dd-93b6-8b8358c36723 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:59.150312 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5860a72e-fc53-45dd-93b6-8b8358c36723 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:59.150585 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5860a72e-fc53-45dd-93b6-8b8358c36723 admin admin] PATH_INFO: `/v3/users/519e93f1ea224168a8cfe171ec9b2a28` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:59.151858 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5860a72e-fc53-45dd-93b6-8b8358c36723 admin admin] RBAC: Authorizing `identity:delete_user(user_id=519e93f1ea224168a8cfe171ec9b2a28)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:59.153837 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5860a72e-fc53-45dd-93b6-8b8358c36723 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:59.342161 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:20:59.366533 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 944/1843] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Sep 23 15:20:58 2022] POST /identity/v3/users => generated 285 bytes in 485 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:20:59.368386 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16062662-23ea-46aa-be8b-58d7db7d196c admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:59.371563 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-16062662-23ea-46aa-be8b-58d7db7d196c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:20:59.385646 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16062662-23ea-46aa-be8b-58d7db7d196c admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:59.391659 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2eb3029-f7a3-4012-b0db-4e9da6490b28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:59.392545 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2eb3029-f7a3-4012-b0db-4e9da6490b28 admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:59.393039 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2eb3029-f7a3-4012-b0db-4e9da6490b28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:59.393475 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2eb3029-f7a3-4012-b0db-4e9da6490b28 admin admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:59.395124 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2eb3029-f7a3-4012-b0db-4e9da6490b28 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:59.397152 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2eb3029-f7a3-4012-b0db-4e9da6490b28 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:59.409959 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 945/1844] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Sep 23 15:20:59 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 23 15:20:59.413876 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2eb3029-f7a3-4012-b0db-4e9da6490b28 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:59.424877 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2eb3029-f7a3-4012-b0db-4e9da6490b28 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:59.430852 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593471e3-4edc-487d-8413-9bf795da7312 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:59.431511 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593471e3-4edc-487d-8413-9bf795da7312 admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:59.431791 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593471e3-4edc-487d-8413-9bf795da7312 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:59.432097 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593471e3-4edc-487d-8413-9bf795da7312 admin admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:59.432752 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593471e3-4edc-487d-8413-9bf795da7312 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:59.434342 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593471e3-4edc-487d-8413-9bf795da7312 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:59.449431 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 946/1845] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Sep 23 15:20:59 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 23 15:20:59.453103 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593471e3-4edc-487d-8413-9bf795da7312 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:59.474209 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593471e3-4edc-487d-8413-9bf795da7312 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:59.481028 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859f1080-a778-422e-881e-d212f50496ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:59.481971 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859f1080-a778-422e-881e-d212f50496ec admin admin] REQUEST_METHOD: `PUT` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:59.482253 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859f1080-a778-422e-881e-d212f50496ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:59.482687 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859f1080-a778-422e-881e-d212f50496ec admin admin] PATH_INFO: `/v3/projects/d77dedd4e5b645b1b9a5e4b0e2541e01/users/6314f89b7f74430eacd9fce53130be47/roles/af257be2dee54d6b942f58650196582c` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:59.512017 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859f1080-a778-422e-881e-d212f50496ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=d77dedd4e5b645b1b9a5e4b0e2541e01, user_id=6314f89b7f74430eacd9fce53130be47, role_id=af257be2dee54d6b942f58650196582c)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:59.515431 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859f1080-a778-422e-881e-d212f50496ec admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:59.648197 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-5860a72e-fc53-45dd-93b6-8b8358c36723 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '519e93f1ea224168a8cfe171ec9b2a28'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:59.665728 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-5860a72e-fc53-45dd-93b6-8b8358c36723 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '519e93f1ea224168a8cfe171ec9b2a28'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:59.668146 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 947/1846] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Sep 23 15:20:59 2022] PUT /identity/v3/projects/d77dedd4e5b645b1b9a5e4b0e2541e01/users/6314f89b7f74430eacd9fce53130be47/roles/af257be2dee54d6b942f58650196582c => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:20:59.677060 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-5860a72e-fc53-45dd-93b6-8b8358c36723 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '519e93f1ea224168a8cfe171ec9b2a28'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:59.677977 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1815bb21-54b4-48f1-b3f4-210cf6d682b0 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:59.678270 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1815bb21-54b4-48f1-b3f4-210cf6d682b0 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:59.678549 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1815bb21-54b4-48f1-b3f4-210cf6d682b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:59.708215 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-1815bb21-54b4-48f1-b3f4-210cf6d682b0 None None] MFA Rules not processed for user `6314f89b7f74430eacd9fce53130be47`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:20:59.709447 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-1815bb21-54b4-48f1-b3f4-210cf6d682b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:20:59.775753 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 948/1847] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:20:59 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 15:20:59.791570 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1815bb21-54b4-48f1-b3f4-210cf6d682b0 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:59.792952 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-1815bb21-54b4-48f1-b3f4-210cf6d682b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:20:59.851891 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1815bb21-54b4-48f1-b3f4-210cf6d682b0 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:59.856033 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24d15f5-5d59-4c93-8eb3-cc4d4f94ad66 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feca6278be904ef8b0dc580a67e477f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:59.856790 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24d15f5-5d59-4c93-8eb3-cc4d4f94ad66 service cinder] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:59.857181 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24d15f5-5d59-4c93-8eb3-cc4d4f94ad66 service cinder] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:59.857520 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24d15f5-5d59-4c93-8eb3-cc4d4f94ad66 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:59.869396 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24d15f5-5d59-4c93-8eb3-cc4d4f94ad66 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:59.870919 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24d15f5-5d59-4c93-8eb3-cc4d4f94ad66 service cinder] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:59.875426 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-5860a72e-fc53-45dd-93b6-8b8358c36723 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '519e93f1ea224168a8cfe171ec9b2a28'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:59.879752 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-5860a72e-fc53-45dd-93b6-8b8358c36723 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '519e93f1ea224168a8cfe171ec9b2a28'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:59.889752 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 900/1848] 10.222.0.55 () {58 vars in 1224 bytes} [Fri Sep 23 15:20:59 2022] DELETE /identity/v3/users/519e93f1ea224168a8cfe171ec9b2a28 => generated 0 bytes in 813 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 15:20:59.902991 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 949/1849] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:20:59 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:20:59.936405 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5860a72e-fc53-45dd-93b6-8b8358c36723 admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:59.941468 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-5860a72e-fc53-45dd-93b6-8b8358c36723 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:00.131828 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5860a72e-fc53-45dd-93b6-8b8358c36723 admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:00.144952 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2f23c8-12c1-4a3e-9e98-ba106118ac73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:00.145742 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2f23c8-12c1-4a3e-9e98-ba106118ac73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:00.146093 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2f23c8-12c1-4a3e-9e98-ba106118ac73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:00.146436 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2f23c8-12c1-4a3e-9e98-ba106118ac73 admin admin] PATH_INFO: `/v3/users/6314f89b7f74430eacd9fce53130be47` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:00.147740 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2f23c8-12c1-4a3e-9e98-ba106118ac73 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6314f89b7f74430eacd9fce53130be47)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:00.150587 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2f23c8-12c1-4a3e-9e98-ba106118ac73 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:00.573648 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-1f2f23c8-12c1-4a3e-9e98-ba106118ac73 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6314f89b7f74430eacd9fce53130be47'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:00.586141 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-1f2f23c8-12c1-4a3e-9e98-ba106118ac73 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6314f89b7f74430eacd9fce53130be47'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:00.594152 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-1f2f23c8-12c1-4a3e-9e98-ba106118ac73 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6314f89b7f74430eacd9fce53130be47'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:00.664096 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24d15f5-5d59-4c93-8eb3-cc4d4f94ad66 service cinder] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:00.693943 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-1f2f23c8-12c1-4a3e-9e98-ba106118ac73 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6314f89b7f74430eacd9fce53130be47'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:00.694572 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-1f2f23c8-12c1-4a3e-9e98-ba106118ac73 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6314f89b7f74430eacd9fce53130be47'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:00.709857 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 901/1850] 10.222.0.55 () {58 vars in 1224 bytes} [Fri Sep 23 15:20:59 2022] DELETE /identity/v3/users/6314f89b7f74430eacd9fce53130be47 => generated 0 bytes in 767 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:21:00.712736 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e7c696-aa5f-4a5a-9301-a24ee5d1f96e None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:00.713128 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e7c696-aa5f-4a5a-9301-a24ee5d1f96e None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:00.713502 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e7c696-aa5f-4a5a-9301-a24ee5d1f96e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:00.735552 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-e4e7c696-aa5f-4a5a-9301-a24ee5d1f96e None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:21:00.736935 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-e4e7c696-aa5f-4a5a-9301-a24ee5d1f96e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:00.763177 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24d15f5-5d59-4c93-8eb3-cc4d4f94ad66 service cinder] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:00.810929 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 902/1851] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:21:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 15:21:00.831494 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e7c696-aa5f-4a5a-9301-a24ee5d1f96e None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:00.835735 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-e4e7c696-aa5f-4a5a-9301-a24ee5d1f96e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:00.865266 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d79c1c-8280-45f8-9657-e4cd3320fd18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:00.866941 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d79c1c-8280-45f8-9657-e4cd3320fd18 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:00.867240 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d79c1c-8280-45f8-9657-e4cd3320fd18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:00.867516 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d79c1c-8280-45f8-9657-e4cd3320fd18 admin admin] PATH_INFO: `/v3/projects/04abaf80888047faad3c392c2d0c08c5` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:00.869799 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d79c1c-8280-45f8-9657-e4cd3320fd18 admin admin] RBAC: Authorizing `identity:delete_project(project_id=04abaf80888047faad3c392c2d0c08c5)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:00.871931 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d79c1c-8280-45f8-9657-e4cd3320fd18 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:00.902911 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e7c696-aa5f-4a5a-9301-a24ee5d1f96e None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:00.910345 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c2d3d1-9094-4076-b14e-53c0126dbd68 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd259178570864ffeb936ee10dfec6b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:00.911079 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c2d3d1-9094-4076-b14e-53c0126dbd68 service neutron] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:00.911457 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c2d3d1-9094-4076-b14e-53c0126dbd68 service neutron] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:00.911829 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c2d3d1-9094-4076-b14e-53c0126dbd68 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:00.926384 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c2d3d1-9094-4076-b14e-53c0126dbd68 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:00.928219 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c2d3d1-9094-4076-b14e-53c0126dbd68 service neutron] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:00.942541 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 903/1852] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:21:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 23 15:21:01.008312 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-67d79c1c-8280-45f8-9657-e4cd3320fd18 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '04abaf80888047faad3c392c2d0c08c5', 'request_id': 'req-67d79c1c-8280-45f8-9657-e4cd3320fd18'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:01.015691 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-67d79c1c-8280-45f8-9657-e4cd3320fd18 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '04abaf80888047faad3c392c2d0c08c5', 'request_id': 'req-67d79c1c-8280-45f8-9657-e4cd3320fd18'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:01.021716 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-67d79c1c-8280-45f8-9657-e4cd3320fd18 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '04abaf80888047faad3c392c2d0c08c5', 'request_id': 'req-67d79c1c-8280-45f8-9657-e4cd3320fd18'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:01.131611 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-67d79c1c-8280-45f8-9657-e4cd3320fd18 admin admin] The token cache is being invalidate because project 04abaf80888047faad3c392c2d0c08c5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81994) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 15:21:01.131904 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-67d79c1c-8280-45f8-9657-e4cd3320fd18 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:01.132597 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-67d79c1c-8280-45f8-9657-e4cd3320fd18 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:01.135438 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 950/1853] 10.222.0.55 () {58 vars in 1233 bytes} [Fri Sep 23 15:21:00 2022] DELETE /identity/v3/projects/04abaf80888047faad3c392c2d0c08c5 => generated 0 bytes in 477 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 15:21:01.140458 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30778ebb-49be-4736-a5fc-23d248ba9012 None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:01.140780 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30778ebb-49be-4736-a5fc-23d248ba9012 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:01.141056 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30778ebb-49be-4736-a5fc-23d248ba9012 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:01.162107 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-30778ebb-49be-4736-a5fc-23d248ba9012 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:21:01.163307 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-30778ebb-49be-4736-a5fc-23d248ba9012 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:01.232786 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 904/1854] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:21:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:21:01.240664 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e60ae54-31f7-45a6-aa68-35314acf9b7d None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:01.240952 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e60ae54-31f7-45a6-aa68-35314acf9b7d None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:01.241232 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e60ae54-31f7-45a6-aa68-35314acf9b7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:01.294625 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-3e60ae54-31f7-45a6-aa68-35314acf9b7d None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:21:01.295963 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-3e60ae54-31f7-45a6-aa68-35314acf9b7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:01.306861 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 951/1855] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:21:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:21:01.318327 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30778ebb-49be-4736-a5fc-23d248ba9012 None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:01.357736 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30778ebb-49be-4736-a5fc-23d248ba9012 None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:01.372752 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f82729-4f2e-4ede-a316-338d777e6655 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:01.373395 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f82729-4f2e-4ede-a316-338d777e6655 admin admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:01.373677 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f82729-4f2e-4ede-a316-338d777e6655 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:01.373948 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f82729-4f2e-4ede-a316-338d777e6655 admin admin] PATH_INFO: `/v3/domains` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:01.374606 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f82729-4f2e-4ede-a316-338d777e6655 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:21:01.375208 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f82729-4f2e-4ede-a316-338d777e6655 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:01.376885 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f82729-4f2e-4ede-a316-338d777e6655 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:01.396826 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 905/1856] 10.222.0.55 () {58 vars in 1153 bytes} [Fri Sep 23 15:21:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:21:01.398602 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e60ae54-31f7-45a6-aa68-35314acf9b7d None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:01.410201 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e60ae54-31f7-45a6-aa68-35314acf9b7d None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:01.416719 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6f7d46-5fdb-4750-be7f-90b0b53dbf40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:01.417662 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6f7d46-5fdb-4750-be7f-90b0b53dbf40 admin admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:01.418357 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6f7d46-5fdb-4750-be7f-90b0b53dbf40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:01.418785 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6f7d46-5fdb-4750-be7f-90b0b53dbf40 admin admin] PATH_INFO: `/v3/projects` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:01.419633 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6f7d46-5fdb-4750-be7f-90b0b53dbf40 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:01.422387 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6f7d46-5fdb-4750-be7f-90b0b53dbf40 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:01.446195 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f82729-4f2e-4ede-a316-338d777e6655 admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:01.448367 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-f5f82729-4f2e-4ede-a316-338d777e6655 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:01.503714 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f82729-4f2e-4ede-a316-338d777e6655 admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:01.579413 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4299b74e-77ed-4089-bcf3-ae7058709811 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:01.579413 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4299b74e-77ed-4089-bcf3-ae7058709811 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:01.579413 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4299b74e-77ed-4089-bcf3-ae7058709811 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:01.579413 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4299b74e-77ed-4089-bcf3-ae7058709811 admin admin] PATH_INFO: `/v3/projects/d77dedd4e5b645b1b9a5e4b0e2541e01` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:01.580464 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4299b74e-77ed-4089-bcf3-ae7058709811 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d77dedd4e5b645b1b9a5e4b0e2541e01)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:01.580464 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4299b74e-77ed-4089-bcf3-ae7058709811 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:01.711055 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 952/1857] 10.222.0.55 () {60 vars in 1153 bytes} [Fri Sep 23 15:21:01 2022] POST /identity/v3/projects => generated 385 bytes in 308 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:21:01.714486 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6f7d46-5fdb-4750-be7f-90b0b53dbf40 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:01.777517 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-4299b74e-77ed-4089-bcf3-ae7058709811 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd77dedd4e5b645b1b9a5e4b0e2541e01', 'request_id': 'req-4299b74e-77ed-4089-bcf3-ae7058709811'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:01.787093 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-4299b74e-77ed-4089-bcf3-ae7058709811 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd77dedd4e5b645b1b9a5e4b0e2541e01', 'request_id': 'req-4299b74e-77ed-4089-bcf3-ae7058709811'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:01.787440 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6f7d46-5fdb-4750-be7f-90b0b53dbf40 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:01.851880 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc62ef6-a3c4-4f0e-85d2-607c12d33f32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:01.852545 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc62ef6-a3c4-4f0e-85d2-607c12d33f32 admin admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:01.852829 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc62ef6-a3c4-4f0e-85d2-607c12d33f32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:01.853599 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc62ef6-a3c4-4f0e-85d2-607c12d33f32 admin admin] PATH_INFO: `/v3/users` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:01.854239 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc62ef6-a3c4-4f0e-85d2-607c12d33f32 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:01.856578 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc62ef6-a3c4-4f0e-85d2-607c12d33f32 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:01.857520 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:21:01.910394 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-4299b74e-77ed-4089-bcf3-ae7058709811 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd77dedd4e5b645b1b9a5e4b0e2541e01', 'request_id': 'req-4299b74e-77ed-4089-bcf3-ae7058709811'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:01.922998 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-4299b74e-77ed-4089-bcf3-ae7058709811 admin admin] The token cache is being invalidate because project d77dedd4e5b645b1b9a5e4b0e2541e01 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81993) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 15:21:01.923528 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-4299b74e-77ed-4089-bcf3-ae7058709811 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:01.924197 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-4299b74e-77ed-4089-bcf3-ae7058709811 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:01.925801 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 906/1858] 10.222.0.55 () {58 vars in 1233 bytes} [Fri Sep 23 15:21:01 2022] DELETE /identity/v3/projects/d77dedd4e5b645b1b9a5e4b0e2541e01 => generated 0 bytes in 481 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:21:01.930312 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97966b5-7d38-4640-a434-d667880e7ac9 None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:01.930608 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97966b5-7d38-4640-a434-d667880e7ac9 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:01.930886 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97966b5-7d38-4640-a434-d667880e7ac9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:01.951848 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-b97966b5-7d38-4640-a434-d667880e7ac9 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:21:01.953116 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-b97966b5-7d38-4640-a434-d667880e7ac9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:01.961541 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 907/1859] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:21:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 15:21:01.969854 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270691d9-be22-4715-acf5-af550ec92a94 None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:01.970141 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270691d9-be22-4715-acf5-af550ec92a94 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:01.970416 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270691d9-be22-4715-acf5-af550ec92a94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:01.991825 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-270691d9-be22-4715-acf5-af550ec92a94 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:21:01.993075 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-270691d9-be22-4715-acf5-af550ec92a94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:02.005681 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 953/1860] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Sep 23 15:21:01 2022] POST /identity/v3/users => generated 293 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 23 15:21:02.006716 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 908/1861] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:21:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 15:21:02.013850 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270691d9-be22-4715-acf5-af550ec92a94 None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:02.014195 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc62ef6-a3c4-4f0e-85d2-607c12d33f32 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:02.016025 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-ffc62ef6-a3c4-4f0e-85d2-607c12d33f32 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:02.024371 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270691d9-be22-4715-acf5-af550ec92a94 None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:02.031313 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc62ef6-a3c4-4f0e-85d2-607c12d33f32 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:02.034726 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a54722-1a71-4247-8ff2-1cb8e895f1cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:02.035419 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a54722-1a71-4247-8ff2-1cb8e895f1cb admin admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:02.035772 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a54722-1a71-4247-8ff2-1cb8e895f1cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:02.036274 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a54722-1a71-4247-8ff2-1cb8e895f1cb admin admin] PATH_INFO: `/v3/domains` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:02.036741 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4d82dd-cf94-4d05-a1f4-e15ed47667ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:02.036957 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a54722-1a71-4247-8ff2-1cb8e895f1cb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:21:02.037281 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a54722-1a71-4247-8ff2-1cb8e895f1cb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:02.037537 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4d82dd-cf94-4d05-a1f4-e15ed47667ce admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:02.037980 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4d82dd-cf94-4d05-a1f4-e15ed47667ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:02.038320 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4d82dd-cf94-4d05-a1f4-e15ed47667ce admin admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:02.039049 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4d82dd-cf94-4d05-a1f4-e15ed47667ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:02.039165 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a54722-1a71-4247-8ff2-1cb8e895f1cb admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:02.040818 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4d82dd-cf94-4d05-a1f4-e15ed47667ce admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:02.053056 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 954/1862] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Sep 23 15:21:02 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:21:02.053387 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 909/1863] 10.222.0.55 () {58 vars in 1153 bytes} [Fri Sep 23 15:21:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:21:02.058536 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4d82dd-cf94-4d05-a1f4-e15ed47667ce admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:02.069017 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a54722-1a71-4247-8ff2-1cb8e895f1cb admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:02.070383 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4d82dd-cf94-4d05-a1f4-e15ed47667ce admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:02.081685 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecab4444-15fd-4b5b-a80f-1fc90ca7875d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:02.081685 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecab4444-15fd-4b5b-a80f-1fc90ca7875d admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:02.081685 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecab4444-15fd-4b5b-a80f-1fc90ca7875d admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:02.081685 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecab4444-15fd-4b5b-a80f-1fc90ca7875d admin admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:02.082149 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecab4444-15fd-4b5b-a80f-1fc90ca7875d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:02.082149 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecab4444-15fd-4b5b-a80f-1fc90ca7875d admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:02.089217 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a54722-1a71-4247-8ff2-1cb8e895f1cb admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:02.091776 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 955/1864] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Sep 23 15:21:02 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:21:02.096088 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecab4444-15fd-4b5b-a80f-1fc90ca7875d admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:02.098084 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f4a7b2-c623-40c7-a9f0-380826b04bc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:02.098719 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f4a7b2-c623-40c7-a9f0-380826b04bc0 admin admin] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:02.098988 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f4a7b2-c623-40c7-a9f0-380826b04bc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:02.099258 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f4a7b2-c623-40c7-a9f0-380826b04bc0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:02.099956 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f4a7b2-c623-40c7-a9f0-380826b04bc0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:02.101892 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f4a7b2-c623-40c7-a9f0-380826b04bc0 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:02.116184 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecab4444-15fd-4b5b-a80f-1fc90ca7875d admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:02.116184 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74a87c1-3330-4c41-9f0a-fb80baa32f66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:02.116184 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74a87c1-3330-4c41-9f0a-fb80baa32f66 admin admin] REQUEST_METHOD: `PUT` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:02.116184 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74a87c1-3330-4c41-9f0a-fb80baa32f66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:02.116547 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74a87c1-3330-4c41-9f0a-fb80baa32f66 admin admin] PATH_INFO: `/v3/projects/d8d751e3cae043ef90bacc4bcd41fdb5/users/8fe5764172834d38bc4fb3be0bf2cd08/roles/af257be2dee54d6b942f58650196582c` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:02.129173 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74a87c1-3330-4c41-9f0a-fb80baa32f66 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8d751e3cae043ef90bacc4bcd41fdb5, user_id=8fe5764172834d38bc4fb3be0bf2cd08, role_id=af257be2dee54d6b942f58650196582c)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:02.131882 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74a87c1-3330-4c41-9f0a-fb80baa32f66 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:02.260219 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 956/1865] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Sep 23 15:21:02 2022] PUT /identity/v3/projects/d8d751e3cae043ef90bacc4bcd41fdb5/users/8fe5764172834d38bc4fb3be0bf2cd08/roles/af257be2dee54d6b942f58650196582c => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:21:02.264120 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 910/1866] 10.222.0.55 () {60 vars in 1153 bytes} [Fri Sep 23 15:21:02 2022] POST /identity/v3/projects => generated 359 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:21:02.271494 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74a87c1-3330-4c41-9f0a-fb80baa32f66 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:02.276545 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d812754-fdcb-4f26-8f42-1323b61b9dd7 None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:02.276835 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d812754-fdcb-4f26-8f42-1323b61b9dd7 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:02.277109 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d812754-fdcb-4f26-8f42-1323b61b9dd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:02.308155 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-4d812754-fdcb-4f26-8f42-1323b61b9dd7 None None] MFA Rules not processed for user `8fe5764172834d38bc4fb3be0bf2cd08`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:21:02.309481 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-4d812754-fdcb-4f26-8f42-1323b61b9dd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:02.332242 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:21:02.340746 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74a87c1-3330-4c41-9f0a-fb80baa32f66 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:02.378890 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 911/1867] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:21:02 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 15:21:02.383447 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d812754-fdcb-4f26-8f42-1323b61b9dd7 None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:02.400059 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14920845-b9ff-4bff-84eb-6dcbe72e954c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:02.400877 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14920845-b9ff-4bff-84eb-6dcbe72e954c admin admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:02.401219 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14920845-b9ff-4bff-84eb-6dcbe72e954c admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:02.401781 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14920845-b9ff-4bff-84eb-6dcbe72e954c admin admin] PATH_INFO: `/v3/users` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:02.402134 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14920845-b9ff-4bff-84eb-6dcbe72e954c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:02.404128 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14920845-b9ff-4bff-84eb-6dcbe72e954c admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:02.447559 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d812754-fdcb-4f26-8f42-1323b61b9dd7 None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:02.452513 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d0689d-a9b0-4639-ac3a-80831f8eee32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:02.453118 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d0689d-a9b0-4639-ac3a-80831f8eee32 admin admin] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:02.453418 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d0689d-a9b0-4639-ac3a-80831f8eee32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:02.453689 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d0689d-a9b0-4639-ac3a-80831f8eee32 admin admin] PATH_INFO: `/v3/projects` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:02.454315 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d0689d-a9b0-4639-ac3a-80831f8eee32 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:02.456235 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d0689d-a9b0-4639-ac3a-80831f8eee32 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:02.589046 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 957/1868] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Sep 23 15:21:02 2022] POST /identity/v3/users => generated 280 bytes in 315 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 23 15:21:02.590848 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14920845-b9ff-4bff-84eb-6dcbe72e954c admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:02.603672 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14920845-b9ff-4bff-84eb-6dcbe72e954c admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:02.608674 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 912/1869] 10.222.0.55 () {60 vars in 1153 bytes} [Fri Sep 23 15:21:02 2022] POST /identity/v3/projects => generated 385 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 23 15:21:02.609600 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bd6c27-6cc2-45b8-8ef8-3f8c2fb0af24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:02.610323 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bd6c27-6cc2-45b8-8ef8-3f8c2fb0af24 admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:02.610713 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bd6c27-6cc2-45b8-8ef8-3f8c2fb0af24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:02.611159 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bd6c27-6cc2-45b8-8ef8-3f8c2fb0af24 admin admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:02.611961 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bd6c27-6cc2-45b8-8ef8-3f8c2fb0af24 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:02.612117 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d0689d-a9b0-4639-ac3a-80831f8eee32 admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:02.614041 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bd6c27-6cc2-45b8-8ef8-3f8c2fb0af24 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:02.627869 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 958/1870] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Sep 23 15:21:02 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:21:02.631796 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bd6c27-6cc2-45b8-8ef8-3f8c2fb0af24 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:02.642496 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bd6c27-6cc2-45b8-8ef8-3f8c2fb0af24 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:02.715647 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d0689d-a9b0-4639-ac3a-80831f8eee32 admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:02.716836 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9779c0-d6e1-4be5-b0a3-50af04800bad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:02.718367 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9779c0-d6e1-4be5-b0a3-50af04800bad admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:02.718692 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9779c0-d6e1-4be5-b0a3-50af04800bad admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:02.718973 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9779c0-d6e1-4be5-b0a3-50af04800bad admin admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:02.719645 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9779c0-d6e1-4be5-b0a3-50af04800bad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:02.721360 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9779c0-d6e1-4be5-b0a3-50af04800bad admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:02.729228 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0ebb63-006f-4e86-ab24-3f1dba5e352e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:02.729228 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0ebb63-006f-4e86-ab24-3f1dba5e352e admin admin] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:02.729555 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0ebb63-006f-4e86-ab24-3f1dba5e352e admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:02.729727 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0ebb63-006f-4e86-ab24-3f1dba5e352e admin admin] PATH_INFO: `/v3/users` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:02.730452 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0ebb63-006f-4e86-ab24-3f1dba5e352e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:02.732530 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0ebb63-006f-4e86-ab24-3f1dba5e352e admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:02.741644 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 959/1871] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Sep 23 15:21:02 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 23 15:21:02.742715 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9779c0-d6e1-4be5-b0a3-50af04800bad admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:02.754004 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9779c0-d6e1-4be5-b0a3-50af04800bad admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:02.758659 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53cc15d-dbc3-449a-852f-7acc042da1b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:02.759364 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53cc15d-dbc3-449a-852f-7acc042da1b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:02.759628 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53cc15d-dbc3-449a-852f-7acc042da1b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:02.759904 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53cc15d-dbc3-449a-852f-7acc042da1b3 admin admin] PATH_INFO: `/v3/projects/915caee612ba4b8fae2fcf3fd39ff4d2/users/36d223cf965f45169085c2451a94c567/roles/af257be2dee54d6b942f58650196582c` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:02.778423 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53cc15d-dbc3-449a-852f-7acc042da1b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=915caee612ba4b8fae2fcf3fd39ff4d2, user_id=36d223cf965f45169085c2451a94c567, role_id=af257be2dee54d6b942f58650196582c)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:02.781008 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53cc15d-dbc3-449a-852f-7acc042da1b3 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:02.917068 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 913/1872] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Sep 23 15:21:02 2022] POST /identity/v3/users => generated 299 bytes in 305 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 23 15:21:02.921000 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0ebb63-006f-4e86-ab24-3f1dba5e352e admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:02.930813 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0ebb63-006f-4e86-ab24-3f1dba5e352e admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:02.936459 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2019fe85-71d4-415f-8156-4782cc8dd094 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:02.937088 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2019fe85-71d4-415f-8156-4782cc8dd094 admin admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:02.937348 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2019fe85-71d4-415f-8156-4782cc8dd094 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:02.937618 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2019fe85-71d4-415f-8156-4782cc8dd094 admin admin] PATH_INFO: `/v3/roles` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:02.938251 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2019fe85-71d4-415f-8156-4782cc8dd094 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:02.939843 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2019fe85-71d4-415f-8156-4782cc8dd094 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:02.953334 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 914/1873] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Sep 23 15:21:02 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 23 15:21:02.955270 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2019fe85-71d4-415f-8156-4782cc8dd094 admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:02.966749 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 960/1874] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Sep 23 15:21:02 2022] PUT /identity/v3/projects/915caee612ba4b8fae2fcf3fd39ff4d2/users/36d223cf965f45169085c2451a94c567/roles/af257be2dee54d6b942f58650196582c => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:21:02.966978 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2019fe85-71d4-415f-8156-4782cc8dd094 admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:02.977618 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3ca9e8-d50a-4d67-a7da-c1cc0f86aeeb None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:02.977916 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3ca9e8-d50a-4d67-a7da-c1cc0f86aeeb None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:02.981976 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3ca9e8-d50a-4d67-a7da-c1cc0f86aeeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:03.010755 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-7e3ca9e8-d50a-4d67-a7da-c1cc0f86aeeb None None] MFA Rules not processed for user `36d223cf965f45169085c2451a94c567`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:21:03.012190 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-7e3ca9e8-d50a-4d67-a7da-c1cc0f86aeeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:03.039769 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef1cacd-1386-43df-a25b-ff3ff2561d29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:03.041639 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef1cacd-1386-43df-a25b-ff3ff2561d29 admin admin] REQUEST_METHOD: `PUT` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:03.041928 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef1cacd-1386-43df-a25b-ff3ff2561d29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:03.042203 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef1cacd-1386-43df-a25b-ff3ff2561d29 admin admin] PATH_INFO: `/v3/projects/0b29d3f8daff4495a986eb90b2436d9a/users/e7017923425546eab1f1bb598ad05111/roles/8e2919e79df040e8926f21607818b58c` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:03.065264 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef1cacd-1386-43df-a25b-ff3ff2561d29 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b29d3f8daff4495a986eb90b2436d9a, user_id=e7017923425546eab1f1bb598ad05111, role_id=8e2919e79df040e8926f21607818b58c)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:03.068012 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef1cacd-1386-43df-a25b-ff3ff2561d29 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:03.095254 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 961/1875] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:21:02 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 15:21:03.138196 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3ca9e8-d50a-4d67-a7da-c1cc0f86aeeb None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:03.139884 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-7e3ca9e8-d50a-4d67-a7da-c1cc0f86aeeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:03.155320 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 915/1876] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Sep 23 15:21:02 2022] PUT /identity/v3/projects/0b29d3f8daff4495a986eb90b2436d9a/users/e7017923425546eab1f1bb598ad05111/roles/8e2919e79df040e8926f21607818b58c => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 15:21:03.159360 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef1cacd-1386-43df-a25b-ff3ff2561d29 admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:03.210882 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3ca9e8-d50a-4d67-a7da-c1cc0f86aeeb None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:03.238745 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef1cacd-1386-43df-a25b-ff3ff2561d29 admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:03.245341 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef34b6b0-55a9-48be-9980-eb8c6f6cf542 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:03.246135 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef34b6b0-55a9-48be-9980-eb8c6f6cf542 admin admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:03.246476 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef34b6b0-55a9-48be-9980-eb8c6f6cf542 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:03.246814 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef34b6b0-55a9-48be-9980-eb8c6f6cf542 admin admin] PATH_INFO: `/v3/roles` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:03.247533 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef34b6b0-55a9-48be-9980-eb8c6f6cf542 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:03.249233 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef34b6b0-55a9-48be-9980-eb8c6f6cf542 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:03.261069 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 916/1877] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Sep 23 15:21:03 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:21:03.265626 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef34b6b0-55a9-48be-9980-eb8c6f6cf542 admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:03.274259 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38826d95-08d3-4e6d-9b0e-be07316fed4c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feca6278be904ef8b0dc580a67e477f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:03.275432 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38826d95-08d3-4e6d-9b0e-be07316fed4c service cinder] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:03.275718 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38826d95-08d3-4e6d-9b0e-be07316fed4c service cinder] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:03.276002 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38826d95-08d3-4e6d-9b0e-be07316fed4c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:03.277344 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef34b6b0-55a9-48be-9980-eb8c6f6cf542 admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:03.283109 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48f859c-b68b-4cca-9d6b-62209f351dd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:03.283710 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48f859c-b68b-4cca-9d6b-62209f351dd9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:03.284001 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48f859c-b68b-4cca-9d6b-62209f351dd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:03.284304 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48f859c-b68b-4cca-9d6b-62209f351dd9 admin admin] PATH_INFO: `/v3/projects/0b29d3f8daff4495a986eb90b2436d9a/users/e7017923425546eab1f1bb598ad05111/roles/8e2919e79df040e8926f21607818b58c` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:03.286322 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48f859c-b68b-4cca-9d6b-62209f351dd9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b29d3f8daff4495a986eb90b2436d9a, user_id=e7017923425546eab1f1bb598ad05111, role_id=8e2919e79df040e8926f21607818b58c)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:03.288905 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48f859c-b68b-4cca-9d6b-62209f351dd9 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:03.312099 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 917/1878] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Sep 23 15:21:03 2022] PUT /identity/v3/projects/0b29d3f8daff4495a986eb90b2436d9a/users/e7017923425546eab1f1bb598ad05111/roles/8e2919e79df040e8926f21607818b58c => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 15:21:03.317483 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9c0d0e-a438-4949-a89b-590cde240e8b None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:03.317772 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9c0d0e-a438-4949-a89b-590cde240e8b None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:03.318051 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9c0d0e-a438-4949-a89b-590cde240e8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:03.341363 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38826d95-08d3-4e6d-9b0e-be07316fed4c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:03.342834 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38826d95-08d3-4e6d-9b0e-be07316fed4c service cinder] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:03.347200 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-2f9c0d0e-a438-4949-a89b-590cde240e8b None None] MFA Rules not processed for user `e7017923425546eab1f1bb598ad05111`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:21:03.348464 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-2f9c0d0e-a438-4949-a89b-590cde240e8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:03.351714 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 962/1879] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:21:03 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 23 15:21:03.367697 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38826d95-08d3-4e6d-9b0e-be07316fed4c service cinder] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:03.378460 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38826d95-08d3-4e6d-9b0e-be07316fed4c service cinder] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:03.421847 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 918/1880] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:21:03 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 15:21:03.433589 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9c0d0e-a438-4949-a89b-590cde240e8b None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:03.438516 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0634e6-01b4-4ac5-aaa0-6c737a5c62d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:03.439250 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0634e6-01b4-4ac5-aaa0-6c737a5c62d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:03.439534 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0634e6-01b4-4ac5-aaa0-6c737a5c62d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:03.439810 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0634e6-01b4-4ac5-aaa0-6c737a5c62d1 admin admin] PATH_INFO: `/v3/users/36d223cf965f45169085c2451a94c567` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:03.441103 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0634e6-01b4-4ac5-aaa0-6c737a5c62d1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=36d223cf965f45169085c2451a94c567)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:03.443130 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0634e6-01b4-4ac5-aaa0-6c737a5c62d1 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:03.489817 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9c0d0e-a438-4949-a89b-590cde240e8b None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:03.493450 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ee81f3-581f-4ba7-a99e-8f9a3c03f1aa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feca6278be904ef8b0dc580a67e477f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:03.494601 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ee81f3-581f-4ba7-a99e-8f9a3c03f1aa service cinder] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:03.494890 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ee81f3-581f-4ba7-a99e-8f9a3c03f1aa service cinder] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:03.495189 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ee81f3-581f-4ba7-a99e-8f9a3c03f1aa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:03.504644 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ee81f3-581f-4ba7-a99e-8f9a3c03f1aa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:03.506110 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ee81f3-581f-4ba7-a99e-8f9a3c03f1aa service cinder] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:03.517030 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 919/1881] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:21:03 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 23 15:21:03.546771 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ee81f3-581f-4ba7-a99e-8f9a3c03f1aa service cinder] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:03.557199 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ee81f3-581f-4ba7-a99e-8f9a3c03f1aa service cinder] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:03.561853 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a029b8d2-fa9c-4b2b-b58d-2d96375d7261 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:03.562471 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a029b8d2-fa9c-4b2b-b58d-2d96375d7261 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:03.562746 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a029b8d2-fa9c-4b2b-b58d-2d96375d7261 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:03.563016 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a029b8d2-fa9c-4b2b-b58d-2d96375d7261 admin admin] PATH_INFO: `/v3/users/8fe5764172834d38bc4fb3be0bf2cd08` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:03.564162 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a029b8d2-fa9c-4b2b-b58d-2d96375d7261 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8fe5764172834d38bc4fb3be0bf2cd08)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:03.566336 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a029b8d2-fa9c-4b2b-b58d-2d96375d7261 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:03.806867 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-db0634e6-01b4-4ac5-aaa0-6c737a5c62d1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '36d223cf965f45169085c2451a94c567'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:03.820735 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-db0634e6-01b4-4ac5-aaa0-6c737a5c62d1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '36d223cf965f45169085c2451a94c567'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:03.831461 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-db0634e6-01b4-4ac5-aaa0-6c737a5c62d1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '36d223cf965f45169085c2451a94c567'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:03.848446 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-a029b8d2-fa9c-4b2b-b58d-2d96375d7261 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8fe5764172834d38bc4fb3be0bf2cd08'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:03.873269 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-a029b8d2-fa9c-4b2b-b58d-2d96375d7261 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8fe5764172834d38bc4fb3be0bf2cd08'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:03.880746 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-a029b8d2-fa9c-4b2b-b58d-2d96375d7261 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8fe5764172834d38bc4fb3be0bf2cd08'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:04.001377 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-db0634e6-01b4-4ac5-aaa0-6c737a5c62d1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '36d223cf965f45169085c2451a94c567'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:04.001377 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-db0634e6-01b4-4ac5-aaa0-6c737a5c62d1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '36d223cf965f45169085c2451a94c567'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:04.004225 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 963/1882] 10.222.0.55 () {58 vars in 1224 bytes} [Fri Sep 23 15:21:03 2022] DELETE /identity/v3/users/36d223cf965f45169085c2451a94c567 => generated 0 bytes in 635 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 15:21:04.007736 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9000d43e-d977-45cf-838f-241c29982a7b None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:04.008059 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9000d43e-d977-45cf-838f-241c29982a7b None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:04.008339 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9000d43e-d977-45cf-838f-241c29982a7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:04.028080 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-9000d43e-d977-45cf-838f-241c29982a7b None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:21:04.029299 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-9000d43e-d977-45cf-838f-241c29982a7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:04.107461 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 964/1883] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:21:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 15:21:04.117160 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9000d43e-d977-45cf-838f-241c29982a7b None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:04.119156 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-9000d43e-d977-45cf-838f-241c29982a7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:04.140405 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:21:04.147894 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-a029b8d2-fa9c-4b2b-b58d-2d96375d7261 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8fe5764172834d38bc4fb3be0bf2cd08'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:04.147894 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-a029b8d2-fa9c-4b2b-b58d-2d96375d7261 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8fe5764172834d38bc4fb3be0bf2cd08'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:04.153716 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 920/1884] 10.222.0.55 () {58 vars in 1224 bytes} [Fri Sep 23 15:21:03 2022] DELETE /identity/v3/users/8fe5764172834d38bc4fb3be0bf2cd08 => generated 0 bytes in 607 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 15:21:04.159266 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee04699-099c-4294-a233-164623dc2e0c None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:04.159720 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee04699-099c-4294-a233-164623dc2e0c None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:04.160006 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee04699-099c-4294-a233-164623dc2e0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:04.184910 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-1ee04699-099c-4294-a233-164623dc2e0c None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:21:04.187666 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-1ee04699-099c-4294-a233-164623dc2e0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:04.195286 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9000d43e-d977-45cf-838f-241c29982a7b None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:04.250805 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 921/1885] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:21:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:21:04.260830 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee04699-099c-4294-a233-164623dc2e0c None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:04.264161 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-1ee04699-099c-4294-a233-164623dc2e0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:04.264281 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca57a26-4e60-44c3-80f3-281c82dd7be6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd259178570864ffeb936ee10dfec6b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:04.264281 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca57a26-4e60-44c3-80f3-281c82dd7be6 service neutron] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:04.264497 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca57a26-4e60-44c3-80f3-281c82dd7be6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:04.264707 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca57a26-4e60-44c3-80f3-281c82dd7be6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:04.266453 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-eca57a26-4e60-44c3-80f3-281c82dd7be6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:04.274984 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee04699-099c-4294-a233-164623dc2e0c None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:04.278632 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba138ba0-1b4f-419d-99da-8724ecfcb683 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd259178570864ffeb936ee10dfec6b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:04.278954 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca57a26-4e60-44c3-80f3-281c82dd7be6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:04.279234 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba138ba0-1b4f-419d-99da-8724ecfcb683 service neutron] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:04.279512 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba138ba0-1b4f-419d-99da-8724ecfcb683 service neutron] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:04.279781 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba138ba0-1b4f-419d-99da-8724ecfcb683 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:04.280457 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca57a26-4e60-44c3-80f3-281c82dd7be6 service neutron] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:04.288820 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba138ba0-1b4f-419d-99da-8724ecfcb683 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:04.290171 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba138ba0-1b4f-419d-99da-8724ecfcb683 service neutron] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:04.291210 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 965/1886] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:21:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 23 15:21:04.304255 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 922/1887] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:21:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 23 15:21:04.573081 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:21:04.963181 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca57a26-4e60-44c3-80f3-281c82dd7be6 service neutron] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:04.964753 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-eca57a26-4e60-44c3-80f3-281c82dd7be6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:04.966959 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba138ba0-1b4f-419d-99da-8724ecfcb683 service neutron] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:04.969610 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-ba138ba0-1b4f-419d-99da-8724ecfcb683 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:04.984239 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca57a26-4e60-44c3-80f3-281c82dd7be6 service neutron] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:04.988349 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba138ba0-1b4f-419d-99da-8724ecfcb683 service neutron] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:04.993596 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1c3c2a-8420-42af-9d45-2f830d35a445 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:04.998089 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1c3c2a-8420-42af-9d45-2f830d35a445 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:04.998089 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1c3c2a-8420-42af-9d45-2f830d35a445 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:04.998089 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1c3c2a-8420-42af-9d45-2f830d35a445 admin admin] PATH_INFO: `/v3/projects/915caee612ba4b8fae2fcf3fd39ff4d2` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:04.998089 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1c3c2a-8420-42af-9d45-2f830d35a445 admin admin] RBAC: Authorizing `identity:delete_project(project_id=915caee612ba4b8fae2fcf3fd39ff4d2)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:04.999006 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1c3c2a-8420-42af-9d45-2f830d35a445 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:05.001797 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445715b8-c662-4ee9-913c-64a190a61497 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:05.002636 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445715b8-c662-4ee9-913c-64a190a61497 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:05.003022 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445715b8-c662-4ee9-913c-64a190a61497 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:05.003773 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445715b8-c662-4ee9-913c-64a190a61497 admin admin] PATH_INFO: `/v3/projects/d8d751e3cae043ef90bacc4bcd41fdb5` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:05.005393 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445715b8-c662-4ee9-913c-64a190a61497 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d8d751e3cae043ef90bacc4bcd41fdb5)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:05.007419 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445715b8-c662-4ee9-913c-64a190a61497 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:05.186161 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-eb1c3c2a-8420-42af-9d45-2f830d35a445 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '915caee612ba4b8fae2fcf3fd39ff4d2', 'request_id': 'req-eb1c3c2a-8420-42af-9d45-2f830d35a445'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:05.219437 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-eb1c3c2a-8420-42af-9d45-2f830d35a445 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '915caee612ba4b8fae2fcf3fd39ff4d2', 'request_id': 'req-eb1c3c2a-8420-42af-9d45-2f830d35a445'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:05.271496 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-445715b8-c662-4ee9-913c-64a190a61497 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd8d751e3cae043ef90bacc4bcd41fdb5', 'request_id': 'req-445715b8-c662-4ee9-913c-64a190a61497'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:05.280434 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-445715b8-c662-4ee9-913c-64a190a61497 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd8d751e3cae043ef90bacc4bcd41fdb5', 'request_id': 'req-445715b8-c662-4ee9-913c-64a190a61497'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:05.287373 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-445715b8-c662-4ee9-913c-64a190a61497 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd8d751e3cae043ef90bacc4bcd41fdb5', 'request_id': 'req-445715b8-c662-4ee9-913c-64a190a61497'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:05.357056 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-eb1c3c2a-8420-42af-9d45-2f830d35a445 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '915caee612ba4b8fae2fcf3fd39ff4d2', 'request_id': 'req-eb1c3c2a-8420-42af-9d45-2f830d35a445'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:05.387845 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-eb1c3c2a-8420-42af-9d45-2f830d35a445 admin admin] The token cache is being invalidate because project 915caee612ba4b8fae2fcf3fd39ff4d2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81993) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 15:21:05.389586 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-eb1c3c2a-8420-42af-9d45-2f830d35a445 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:05.390662 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-eb1c3c2a-8420-42af-9d45-2f830d35a445 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:05.392307 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:21:05.394215 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 923/1888] 10.222.0.55 () {58 vars in 1233 bytes} [Fri Sep 23 15:21:04 2022] DELETE /identity/v3/projects/915caee612ba4b8fae2fcf3fd39ff4d2 => generated 0 bytes in 428 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 15:21:05.398791 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac61c097-f062-4442-9bee-62992cbc753a None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:05.399361 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac61c097-f062-4442-9bee-62992cbc753a None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:05.399726 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac61c097-f062-4442-9bee-62992cbc753a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:05.422279 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-ac61c097-f062-4442-9bee-62992cbc753a None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:21:05.427792 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-ac61c097-f062-4442-9bee-62992cbc753a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:05.487698 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 924/1889] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:21:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 15:21:05.497682 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a340901e-ad4c-42bb-a5cf-2d21608a3652 None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:05.497974 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a340901e-ad4c-42bb-a5cf-2d21608a3652 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:05.498453 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a340901e-ad4c-42bb-a5cf-2d21608a3652 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:05.499389 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-445715b8-c662-4ee9-913c-64a190a61497 admin admin] The token cache is being invalidate because project d8d751e3cae043ef90bacc4bcd41fdb5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81994) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 15:21:05.499887 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-445715b8-c662-4ee9-913c-64a190a61497 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:05.500878 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-445715b8-c662-4ee9-913c-64a190a61497 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:05.513215 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 966/1890] 10.222.0.55 () {58 vars in 1233 bytes} [Fri Sep 23 15:21:04 2022] DELETE /identity/v3/projects/d8d751e3cae043ef90bacc4bcd41fdb5 => generated 0 bytes in 541 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:21:05.529089 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-a340901e-ad4c-42bb-a5cf-2d21608a3652 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:21:05.529256 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445715b8-c662-4ee9-913c-64a190a61497 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:05.530625 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-445715b8-c662-4ee9-913c-64a190a61497 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:05.532469 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-a340901e-ad4c-42bb-a5cf-2d21608a3652 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:05.557914 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 925/1891] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:21:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:21:05.558492 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a340901e-ad4c-42bb-a5cf-2d21608a3652 None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:05.560923 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445715b8-c662-4ee9-913c-64a190a61497 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:05.571324 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75361342-e10f-4c78-b324-288abc3a457a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:05.572042 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75361342-e10f-4c78-b324-288abc3a457a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:05.572320 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75361342-e10f-4c78-b324-288abc3a457a admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:05.572597 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75361342-e10f-4c78-b324-288abc3a457a admin admin] PATH_INFO: `/v3/users/e7017923425546eab1f1bb598ad05111` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:05.573771 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75361342-e10f-4c78-b324-288abc3a457a admin admin] RBAC: Authorizing `identity:delete_user(user_id=e7017923425546eab1f1bb598ad05111)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:05.575884 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75361342-e10f-4c78-b324-288abc3a457a admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:05.576408 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a340901e-ad4c-42bb-a5cf-2d21608a3652 None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:05.582760 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f56048-3e21-4a62-b783-031d62ea65d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:05.588509 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f56048-3e21-4a62-b783-031d62ea65d2 admin admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:05.588509 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f56048-3e21-4a62-b783-031d62ea65d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:05.588509 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f56048-3e21-4a62-b783-031d62ea65d2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:05.588509 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f56048-3e21-4a62-b783-031d62ea65d2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:21:05.588509 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f56048-3e21-4a62-b783-031d62ea65d2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:05.588509 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f56048-3e21-4a62-b783-031d62ea65d2 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:05.603499 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 926/1892] 10.222.0.55 () {58 vars in 1153 bytes} [Fri Sep 23 15:21:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:21:05.608439 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f56048-3e21-4a62-b783-031d62ea65d2 admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:05.625197 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f56048-3e21-4a62-b783-031d62ea65d2 admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:05.630469 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b58b66-29d1-447b-a414-7f24a4176a7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:05.631082 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b58b66-29d1-447b-a414-7f24a4176a7b admin admin] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:05.631366 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b58b66-29d1-447b-a414-7f24a4176a7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:05.631639 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b58b66-29d1-447b-a414-7f24a4176a7b admin admin] PATH_INFO: `/v3/projects` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:05.632316 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b58b66-29d1-447b-a414-7f24a4176a7b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:05.634697 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b58b66-29d1-447b-a414-7f24a4176a7b admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:05.655120 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:21:05.808114 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 927/1893] 10.222.0.55 () {60 vars in 1153 bytes} [Fri Sep 23 15:21:05 2022] POST /identity/v3/projects => generated 375 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 23 15:21:05.809280 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b58b66-29d1-447b-a414-7f24a4176a7b admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:05.895263 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b58b66-29d1-447b-a414-7f24a4176a7b admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:05.905411 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec4a1f4-fb3d-4850-b137-a1a5308ddd8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:05.905411 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec4a1f4-fb3d-4850-b137-a1a5308ddd8c admin admin] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:05.905411 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec4a1f4-fb3d-4850-b137-a1a5308ddd8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:05.905411 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec4a1f4-fb3d-4850-b137-a1a5308ddd8c admin admin] PATH_INFO: `/v3/users` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:05.905939 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec4a1f4-fb3d-4850-b137-a1a5308ddd8c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:05.905939 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec4a1f4-fb3d-4850-b137-a1a5308ddd8c admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:05.919903 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-75361342-e10f-4c78-b324-288abc3a457a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e7017923425546eab1f1bb598ad05111'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:05.933983 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-75361342-e10f-4c78-b324-288abc3a457a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e7017923425546eab1f1bb598ad05111'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:05.941473 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-75361342-e10f-4c78-b324-288abc3a457a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e7017923425546eab1f1bb598ad05111'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:05.998936 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 928/1894] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Sep 23 15:21:05 2022] POST /identity/v3/users => generated 288 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 23 15:21:06.001438 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec4a1f4-fb3d-4850-b137-a1a5308ddd8c admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:06.011867 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec4a1f4-fb3d-4850-b137-a1a5308ddd8c admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:06.018788 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c520e6bf-a467-4cd1-a705-5b04e977e8d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:06.019547 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c520e6bf-a467-4cd1-a705-5b04e977e8d4 admin admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:06.019901 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c520e6bf-a467-4cd1-a705-5b04e977e8d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:06.020310 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c520e6bf-a467-4cd1-a705-5b04e977e8d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:06.021090 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c520e6bf-a467-4cd1-a705-5b04e977e8d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:06.022861 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c520e6bf-a467-4cd1-a705-5b04e977e8d4 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:06.035412 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 929/1895] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Sep 23 15:21:06 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:21:06.039467 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c520e6bf-a467-4cd1-a705-5b04e977e8d4 admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:06.052283 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c520e6bf-a467-4cd1-a705-5b04e977e8d4 admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:06.069351 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d50155e-47d8-470d-9ab5-43410ceae04d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:06.070038 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d50155e-47d8-470d-9ab5-43410ceae04d admin admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:06.070317 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d50155e-47d8-470d-9ab5-43410ceae04d admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:06.070647 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d50155e-47d8-470d-9ab5-43410ceae04d admin admin] PATH_INFO: `/v3/roles` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:06.071320 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d50155e-47d8-470d-9ab5-43410ceae04d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:06.073543 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d50155e-47d8-470d-9ab5-43410ceae04d admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:06.074097 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-75361342-e10f-4c78-b324-288abc3a457a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e7017923425546eab1f1bb598ad05111'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:06.078841 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-75361342-e10f-4c78-b324-288abc3a457a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e7017923425546eab1f1bb598ad05111'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:06.092815 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 967/1896] 10.222.0.55 () {58 vars in 1224 bytes} [Fri Sep 23 15:21:05 2022] DELETE /identity/v3/users/e7017923425546eab1f1bb598ad05111 => generated 0 bytes in 560 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:21:06.100024 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 930/1897] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Sep 23 15:21:06 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 23 15:21:06.104561 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75361342-e10f-4c78-b324-288abc3a457a admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:06.106212 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-75361342-e10f-4c78-b324-288abc3a457a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:06.186873 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75361342-e10f-4c78-b324-288abc3a457a admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:06.193549 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140d2f0b-ae0d-4720-bf86-d9190174b9bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:06.194491 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140d2f0b-ae0d-4720-bf86-d9190174b9bb admin admin] REQUEST_METHOD: `PUT` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:06.194885 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140d2f0b-ae0d-4720-bf86-d9190174b9bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:06.195236 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140d2f0b-ae0d-4720-bf86-d9190174b9bb admin admin] PATH_INFO: `/v3/projects/f8c92ebecfe64b8f96aeac894f6947e0/users/5c857ee2049543479944da34eda79b5c/roles/af257be2dee54d6b942f58650196582c` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:06.224564 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140d2f0b-ae0d-4720-bf86-d9190174b9bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=f8c92ebecfe64b8f96aeac894f6947e0, user_id=5c857ee2049543479944da34eda79b5c, role_id=af257be2dee54d6b942f58650196582c)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:06.230838 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140d2f0b-ae0d-4720-bf86-d9190174b9bb admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:06.384741 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 968/1898] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Sep 23 15:21:06 2022] PUT /identity/v3/projects/f8c92ebecfe64b8f96aeac894f6947e0/users/5c857ee2049543479944da34eda79b5c/roles/af257be2dee54d6b942f58650196582c => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:21:06.397267 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38d4336-1343-4cbd-b5bd-d5f0555d12ab None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:06.397800 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38d4336-1343-4cbd-b5bd-d5f0555d12ab None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:06.398886 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38d4336-1343-4cbd-b5bd-d5f0555d12ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:06.402688 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140d2f0b-ae0d-4720-bf86-d9190174b9bb admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:06.406366 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-140d2f0b-ae0d-4720-bf86-d9190174b9bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:06.452563 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-a38d4336-1343-4cbd-b5bd-d5f0555d12ab None None] MFA Rules not processed for user `5c857ee2049543479944da34eda79b5c`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:21:06.454068 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-a38d4336-1343-4cbd-b5bd-d5f0555d12ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:06.480015 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140d2f0b-ae0d-4720-bf86-d9190174b9bb admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:06.484912 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bc92aa-eab3-4a47-ae97-3ed85df7ede3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:06.485622 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bc92aa-eab3-4a47-ae97-3ed85df7ede3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:06.486055 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bc92aa-eab3-4a47-ae97-3ed85df7ede3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:06.493942 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bc92aa-eab3-4a47-ae97-3ed85df7ede3 admin admin] PATH_INFO: `/v3/users/3ec3bec059b84e52b000ac20f9ef08a6` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:06.493942 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bc92aa-eab3-4a47-ae97-3ed85df7ede3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3ec3bec059b84e52b000ac20f9ef08a6)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:06.493942 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bc92aa-eab3-4a47-ae97-3ed85df7ede3 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:06.520700 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:21:06.536794 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 931/1899] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:21:06 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:21:06.547787 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38d4336-1343-4cbd-b5bd-d5f0555d12ab None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:06.549212 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-a38d4336-1343-4cbd-b5bd-d5f0555d12ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:06.611367 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38d4336-1343-4cbd-b5bd-d5f0555d12ab None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:06.614864 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc58cf7d-84f8-4458-abad-6c695def981a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feca6278be904ef8b0dc580a67e477f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:06.615490 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc58cf7d-84f8-4458-abad-6c695def981a service cinder] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:06.615767 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc58cf7d-84f8-4458-abad-6c695def981a service cinder] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:06.616064 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc58cf7d-84f8-4458-abad-6c695def981a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:06.624639 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc58cf7d-84f8-4458-abad-6c695def981a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:06.626090 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc58cf7d-84f8-4458-abad-6c695def981a service cinder] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:06.637168 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 932/1900] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:21:06 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 23 15:21:06.731670 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc58cf7d-84f8-4458-abad-6c695def981a service cinder] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:06.734579 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-cc58cf7d-84f8-4458-abad-6c695def981a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:06.749487 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc58cf7d-84f8-4458-abad-6c695def981a service cinder] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:06.755189 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f3ff62-5a98-43ab-9935-cd4c59d99092 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:06.758748 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f3ff62-5a98-43ab-9935-cd4c59d99092 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:06.758748 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f3ff62-5a98-43ab-9935-cd4c59d99092 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:06.758748 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f3ff62-5a98-43ab-9935-cd4c59d99092 admin admin] PATH_INFO: `/v3/projects/0b29d3f8daff4495a986eb90b2436d9a` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:06.758748 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f3ff62-5a98-43ab-9935-cd4c59d99092 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b29d3f8daff4495a986eb90b2436d9a)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:06.760913 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f3ff62-5a98-43ab-9935-cd4c59d99092 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:06.825778 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-42bc92aa-eab3-4a47-ae97-3ed85df7ede3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3ec3bec059b84e52b000ac20f9ef08a6'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:06.866640 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-42bc92aa-eab3-4a47-ae97-3ed85df7ede3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3ec3bec059b84e52b000ac20f9ef08a6'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:06.877295 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-42bc92aa-eab3-4a47-ae97-3ed85df7ede3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3ec3bec059b84e52b000ac20f9ef08a6'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:06.960263 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-63f3ff62-5a98-43ab-9935-cd4c59d99092 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b29d3f8daff4495a986eb90b2436d9a', 'request_id': 'req-63f3ff62-5a98-43ab-9935-cd4c59d99092'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:06.970103 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-42bc92aa-eab3-4a47-ae97-3ed85df7ede3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3ec3bec059b84e52b000ac20f9ef08a6'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:06.970976 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-63f3ff62-5a98-43ab-9935-cd4c59d99092 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b29d3f8daff4495a986eb90b2436d9a', 'request_id': 'req-63f3ff62-5a98-43ab-9935-cd4c59d99092'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:06.971362 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-42bc92aa-eab3-4a47-ae97-3ed85df7ede3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3ec3bec059b84e52b000ac20f9ef08a6'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:06.980311 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 969/1901] 10.222.0.55 () {58 vars in 1224 bytes} [Fri Sep 23 15:21:06 2022] DELETE /identity/v3/users/3ec3bec059b84e52b000ac20f9ef08a6 => generated 0 bytes in 578 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 15:21:06.985014 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80749919-cedb-4346-a89f-d9ec187ed011 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:06.985305 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80749919-cedb-4346-a89f-d9ec187ed011 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:06.985589 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80749919-cedb-4346-a89f-d9ec187ed011 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:07.007308 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-80749919-cedb-4346-a89f-d9ec187ed011 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:21:07.009769 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-80749919-cedb-4346-a89f-d9ec187ed011 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:07.083852 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 970/1902] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:21:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 15:21:07.093730 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-63f3ff62-5a98-43ab-9935-cd4c59d99092 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b29d3f8daff4495a986eb90b2436d9a', 'request_id': 'req-63f3ff62-5a98-43ab-9935-cd4c59d99092'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:07.097750 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80749919-cedb-4346-a89f-d9ec187ed011 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:07.100062 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-80749919-cedb-4346-a89f-d9ec187ed011 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:07.120484 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-63f3ff62-5a98-43ab-9935-cd4c59d99092 admin admin] The token cache is being invalidate because project 0b29d3f8daff4495a986eb90b2436d9a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81993) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 15:21:07.120878 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-63f3ff62-5a98-43ab-9935-cd4c59d99092 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:07.121916 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-63f3ff62-5a98-43ab-9935-cd4c59d99092 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:07.123963 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 933/1903] 10.222.0.55 () {58 vars in 1233 bytes} [Fri Sep 23 15:21:06 2022] DELETE /identity/v3/projects/0b29d3f8daff4495a986eb90b2436d9a => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 15:21:07.136952 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b4b427-5aef-40d3-842e-32399dbfd959 None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:07.137488 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b4b427-5aef-40d3-842e-32399dbfd959 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:07.137958 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b4b427-5aef-40d3-842e-32399dbfd959 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:07.178153 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-30b4b427-5aef-40d3-842e-32399dbfd959 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:21:07.179597 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-30b4b427-5aef-40d3-842e-32399dbfd959 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:07.192103 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 934/1904] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:21:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:21:07.199448 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079f0815-4f85-4299-92b0-77e9f31bbc6b None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:07.206103 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079f0815-4f85-4299-92b0-77e9f31bbc6b None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:07.206103 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079f0815-4f85-4299-92b0-77e9f31bbc6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:07.206526 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80749919-cedb-4346-a89f-d9ec187ed011 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:07.208855 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddfa64b-18ed-496d-b2f2-e8a4ae0416a4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd259178570864ffeb936ee10dfec6b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:07.209972 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddfa64b-18ed-496d-b2f2-e8a4ae0416a4 service neutron] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:07.210416 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddfa64b-18ed-496d-b2f2-e8a4ae0416a4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:07.210843 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddfa64b-18ed-496d-b2f2-e8a4ae0416a4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:07.212976 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-cddfa64b-18ed-496d-b2f2-e8a4ae0416a4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:07.226770 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-079f0815-4f85-4299-92b0-77e9f31bbc6b None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:21:07.229193 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-079f0815-4f85-4299-92b0-77e9f31bbc6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:07.232699 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddfa64b-18ed-496d-b2f2-e8a4ae0416a4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:07.235073 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddfa64b-18ed-496d-b2f2-e8a4ae0416a4 service neutron] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:07.242447 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 935/1905] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:21:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 15:21:07.251963 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079f0815-4f85-4299-92b0-77e9f31bbc6b None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:07.254419 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 971/1906] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:21:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:21:07.259078 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079f0815-4f85-4299-92b0-77e9f31bbc6b None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:07.265436 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d6e26d-1f42-47bc-b53f-6f36fca9e302 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:07.266254 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d6e26d-1f42-47bc-b53f-6f36fca9e302 admin admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:07.266685 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d6e26d-1f42-47bc-b53f-6f36fca9e302 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:07.267108 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d6e26d-1f42-47bc-b53f-6f36fca9e302 admin admin] PATH_INFO: `/v3/domains` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:07.267930 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d6e26d-1f42-47bc-b53f-6f36fca9e302 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:21:07.268427 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d6e26d-1f42-47bc-b53f-6f36fca9e302 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:07.270219 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d6e26d-1f42-47bc-b53f-6f36fca9e302 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:07.283589 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 936/1907] 10.222.0.55 () {58 vars in 1153 bytes} [Fri Sep 23 15:21:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:21:07.300202 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddfa64b-18ed-496d-b2f2-e8a4ae0416a4 service neutron] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:07.311569 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddfa64b-18ed-496d-b2f2-e8a4ae0416a4 service neutron] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:07.316878 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd9485f-6afd-4293-a7d4-c51aab9d89fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:07.317582 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd9485f-6afd-4293-a7d4-c51aab9d89fe admin admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:07.318679 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd9485f-6afd-4293-a7d4-c51aab9d89fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:07.318873 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd9485f-6afd-4293-a7d4-c51aab9d89fe admin admin] PATH_INFO: `/v3/projects` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:07.319606 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd9485f-6afd-4293-a7d4-c51aab9d89fe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:07.322247 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd9485f-6afd-4293-a7d4-c51aab9d89fe admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:07.569107 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 972/1908] 10.222.0.55 () {60 vars in 1153 bytes} [Fri Sep 23 15:21:07 2022] POST /identity/v3/projects => generated 385 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:21:07.573162 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d6e26d-1f42-47bc-b53f-6f36fca9e302 admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:07.586708 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d6e26d-1f42-47bc-b53f-6f36fca9e302 admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:07.698055 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe284f4-90b9-448c-94a8-37e3098b595b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:07.698749 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe284f4-90b9-448c-94a8-37e3098b595b admin admin] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:07.699159 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe284f4-90b9-448c-94a8-37e3098b595b admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:07.699525 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe284f4-90b9-448c-94a8-37e3098b595b admin admin] PATH_INFO: `/v3/users` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:07.700621 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe284f4-90b9-448c-94a8-37e3098b595b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:07.702870 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe284f4-90b9-448c-94a8-37e3098b595b admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:07.831945 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 937/1909] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Sep 23 15:21:07 2022] POST /identity/v3/users => generated 293 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 23 15:21:07.843007 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd9485f-6afd-4293-a7d4-c51aab9d89fe admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:07.855632 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd9485f-6afd-4293-a7d4-c51aab9d89fe admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:07.862690 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159beb5e-774e-4282-9910-8377337eb31a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:07.863353 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159beb5e-774e-4282-9910-8377337eb31a admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:07.863634 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159beb5e-774e-4282-9910-8377337eb31a admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:07.864557 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159beb5e-774e-4282-9910-8377337eb31a admin admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:07.865290 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159beb5e-774e-4282-9910-8377337eb31a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:07.867511 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159beb5e-774e-4282-9910-8377337eb31a admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:07.885053 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 973/1910] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Sep 23 15:21:07 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:21:07.888897 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe284f4-90b9-448c-94a8-37e3098b595b admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:07.899486 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe284f4-90b9-448c-94a8-37e3098b595b admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:07.905667 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82825e3-1622-460d-bd47-ff2aba088f17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:07.906354 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82825e3-1622-460d-bd47-ff2aba088f17 admin admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:07.906666 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82825e3-1622-460d-bd47-ff2aba088f17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:07.906943 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82825e3-1622-460d-bd47-ff2aba088f17 admin admin] PATH_INFO: `/v3/roles` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:07.907971 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82825e3-1622-460d-bd47-ff2aba088f17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:07.909249 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82825e3-1622-460d-bd47-ff2aba088f17 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:07.922612 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 938/1911] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Sep 23 15:21:07 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 23 15:21:07.928020 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159beb5e-774e-4282-9910-8377337eb31a admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:07.944289 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159beb5e-774e-4282-9910-8377337eb31a admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:07.966991 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e507ecc0-0f44-4d57-8596-9097715afa74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:07.967951 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e507ecc0-0f44-4d57-8596-9097715afa74 admin admin] REQUEST_METHOD: `PUT` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:07.969188 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e507ecc0-0f44-4d57-8596-9097715afa74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:07.969544 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e507ecc0-0f44-4d57-8596-9097715afa74 admin admin] PATH_INFO: `/v3/projects/249110bfa15441cdb278c5b3d85a49a8/users/faff03e3e5e7483ab8d21aac41447e1e/roles/af257be2dee54d6b942f58650196582c` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:08.002393 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e507ecc0-0f44-4d57-8596-9097715afa74 admin admin] RBAC: Authorizing `identity:create_grant(project_id=249110bfa15441cdb278c5b3d85a49a8, user_id=faff03e3e5e7483ab8d21aac41447e1e, role_id=af257be2dee54d6b942f58650196582c)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:08.005403 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e507ecc0-0f44-4d57-8596-9097715afa74 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:08.067934 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82825e3-1622-460d-bd47-ff2aba088f17 admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:08.073209 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-f82825e3-1622-460d-bd47-ff2aba088f17 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:08.101306 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82825e3-1622-460d-bd47-ff2aba088f17 admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:08.107535 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a0e5a6-244d-4aee-a4e5-c0d508a2f9fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:08.108602 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a0e5a6-244d-4aee-a4e5-c0d508a2f9fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:08.109059 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a0e5a6-244d-4aee-a4e5-c0d508a2f9fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:08.109591 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a0e5a6-244d-4aee-a4e5-c0d508a2f9fe admin admin] PATH_INFO: `/v3/projects/24c3c30a9f2945dc80221892debb75d6` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:08.110926 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a0e5a6-244d-4aee-a4e5-c0d508a2f9fe admin admin] RBAC: Authorizing `identity:delete_project(project_id=24c3c30a9f2945dc80221892debb75d6)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:08.113308 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a0e5a6-244d-4aee-a4e5-c0d508a2f9fe admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:08.169512 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 974/1912] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Sep 23 15:21:07 2022] PUT /identity/v3/projects/249110bfa15441cdb278c5b3d85a49a8/users/faff03e3e5e7483ab8d21aac41447e1e/roles/af257be2dee54d6b942f58650196582c => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:21:08.182911 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beedda95-e002-4061-a461-151d9d94af4b None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:08.183193 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beedda95-e002-4061-a461-151d9d94af4b None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:08.183671 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beedda95-e002-4061-a461-151d9d94af4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:08.217474 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-beedda95-e002-4061-a461-151d9d94af4b None None] MFA Rules not processed for user `faff03e3e5e7483ab8d21aac41447e1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:21:08.218926 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-beedda95-e002-4061-a461-151d9d94af4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:08.262691 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-84a0e5a6-244d-4aee-a4e5-c0d508a2f9fe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '24c3c30a9f2945dc80221892debb75d6', 'request_id': 'req-84a0e5a6-244d-4aee-a4e5-c0d508a2f9fe'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:08.281135 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-84a0e5a6-244d-4aee-a4e5-c0d508a2f9fe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '24c3c30a9f2945dc80221892debb75d6', 'request_id': 'req-84a0e5a6-244d-4aee-a4e5-c0d508a2f9fe'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:08.369682 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 975/1913] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:21:08 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:21:08.371616 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beedda95-e002-4061-a461-151d9d94af4b None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:08.379642 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-84a0e5a6-244d-4aee-a4e5-c0d508a2f9fe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '24c3c30a9f2945dc80221892debb75d6', 'request_id': 'req-84a0e5a6-244d-4aee-a4e5-c0d508a2f9fe'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:08.403251 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-84a0e5a6-244d-4aee-a4e5-c0d508a2f9fe admin admin] The token cache is being invalidate because project 24c3c30a9f2945dc80221892debb75d6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81993) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 15:21:08.404354 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-84a0e5a6-244d-4aee-a4e5-c0d508a2f9fe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:08.404745 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-84a0e5a6-244d-4aee-a4e5-c0d508a2f9fe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:08.410803 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 939/1914] 10.222.0.55 () {58 vars in 1233 bytes} [Fri Sep 23 15:21:08 2022] DELETE /identity/v3/projects/24c3c30a9f2945dc80221892debb75d6 => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:21:08.415967 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a0e5a6-244d-4aee-a4e5-c0d508a2f9fe admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:08.417549 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-84a0e5a6-244d-4aee-a4e5-c0d508a2f9fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:08.480958 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beedda95-e002-4061-a461-151d9d94af4b None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:08.538392 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a0e5a6-244d-4aee-a4e5-c0d508a2f9fe admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:08.545426 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f22053c-a4cf-44ef-9b09-2f67a64206e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:08.546375 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f22053c-a4cf-44ef-9b09-2f67a64206e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:08.546846 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f22053c-a4cf-44ef-9b09-2f67a64206e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:08.547293 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f22053c-a4cf-44ef-9b09-2f67a64206e0 admin admin] PATH_INFO: `/v3/users/8f596be90fd4448cb93eb90578d1563d` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:08.548968 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f22053c-a4cf-44ef-9b09-2f67a64206e0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8f596be90fd4448cb93eb90578d1563d)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:08.551408 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f22053c-a4cf-44ef-9b09-2f67a64206e0 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:08.596996 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fbd065-4953-43c2-a7bd-4e26ed465988 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:08.597651 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fbd065-4953-43c2-a7bd-4e26ed465988 admin admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:08.597938 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fbd065-4953-43c2-a7bd-4e26ed465988 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:08.598241 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fbd065-4953-43c2-a7bd-4e26ed465988 admin admin] PATH_INFO: `/v3/projects` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:08.598896 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fbd065-4953-43c2-a7bd-4e26ed465988 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:08.600998 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fbd065-4953-43c2-a7bd-4e26ed465988 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:08.706433 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 976/1915] 10.222.0.55 () {60 vars in 1153 bytes} [Fri Sep 23 15:21:08 2022] POST /identity/v3/projects => generated 387 bytes in 335 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 23 15:21:08.710547 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fbd065-4953-43c2-a7bd-4e26ed465988 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:08.712389 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-95fbd065-4953-43c2-a7bd-4e26ed465988 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:08.827069 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fbd065-4953-43c2-a7bd-4e26ed465988 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:08.827069 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f9389e-0241-48f5-8099-3b939ff7fc1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:08.827069 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f9389e-0241-48f5-8099-3b939ff7fc1c admin admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:08.827069 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f9389e-0241-48f5-8099-3b939ff7fc1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:08.827419 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f9389e-0241-48f5-8099-3b939ff7fc1c admin admin] PATH_INFO: `/v3/users` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:08.827419 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f9389e-0241-48f5-8099-3b939ff7fc1c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:08.827419 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f9389e-0241-48f5-8099-3b939ff7fc1c admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:08.840820 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-4f22053c-a4cf-44ef-9b09-2f67a64206e0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8f596be90fd4448cb93eb90578d1563d'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:08.876888 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-4f22053c-a4cf-44ef-9b09-2f67a64206e0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8f596be90fd4448cb93eb90578d1563d'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:08.896745 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-4f22053c-a4cf-44ef-9b09-2f67a64206e0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8f596be90fd4448cb93eb90578d1563d'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:08.974268 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 977/1916] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Sep 23 15:21:08 2022] POST /identity/v3/users => generated 300 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 23 15:21:08.978769 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f9389e-0241-48f5-8099-3b939ff7fc1c admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:09.001750 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f9389e-0241-48f5-8099-3b939ff7fc1c admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:09.005444 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-4f22053c-a4cf-44ef-9b09-2f67a64206e0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8f596be90fd4448cb93eb90578d1563d'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:09.006091 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-4f22053c-a4cf-44ef-9b09-2f67a64206e0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8f596be90fd4448cb93eb90578d1563d'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:09.016432 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d93b2f-d1cb-47b7-aed5-9b4d329e922f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:09.020039 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 940/1917] 10.222.0.55 () {58 vars in 1224 bytes} [Fri Sep 23 15:21:08 2022] DELETE /identity/v3/users/8f596be90fd4448cb93eb90578d1563d => generated 0 bytes in 604 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:21:09.023133 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d93b2f-d1cb-47b7-aed5-9b4d329e922f admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:09.023565 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d93b2f-d1cb-47b7-aed5-9b4d329e922f admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:09.023971 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d93b2f-d1cb-47b7-aed5-9b4d329e922f admin admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:09.025309 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d93b2f-d1cb-47b7-aed5-9b4d329e922f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:09.030783 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d93b2f-d1cb-47b7-aed5-9b4d329e922f admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:09.056542 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 978/1918] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Sep 23 15:21:08 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:21:09.060717 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f22053c-a4cf-44ef-9b09-2f67a64206e0 admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:09.062634 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-4f22053c-a4cf-44ef-9b09-2f67a64206e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:09.250098 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f22053c-a4cf-44ef-9b09-2f67a64206e0 admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:09.256504 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99702b8d-cf6f-4b02-8727-0d76bd075b78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:09.257423 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99702b8d-cf6f-4b02-8727-0d76bd075b78 admin admin] REQUEST_METHOD: `PUT` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:09.257935 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99702b8d-cf6f-4b02-8727-0d76bd075b78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:09.258332 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99702b8d-cf6f-4b02-8727-0d76bd075b78 admin admin] PATH_INFO: `/v3/projects/bb7c99deb41246d78733f3e91a015614/users/8ea8ba3e80394fa1b9f5e3b66afe99d4/roles/8e2919e79df040e8926f21607818b58c` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:09.280295 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99702b8d-cf6f-4b02-8727-0d76bd075b78 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb7c99deb41246d78733f3e91a015614, user_id=8ea8ba3e80394fa1b9f5e3b66afe99d4, role_id=8e2919e79df040e8926f21607818b58c)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:09.284103 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99702b8d-cf6f-4b02-8727-0d76bd075b78 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:09.419778 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:21:09.475450 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 941/1919] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Sep 23 15:21:09 2022] PUT /identity/v3/projects/bb7c99deb41246d78733f3e91a015614/users/8ea8ba3e80394fa1b9f5e3b66afe99d4/roles/8e2919e79df040e8926f21607818b58c => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:21:09.480212 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d93b2f-d1cb-47b7-aed5-9b4d329e922f admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:09.493539 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d93b2f-d1cb-47b7-aed5-9b4d329e922f admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:09.570981 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377022bf-2be6-4f99-b9ac-2956c85b275e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:09.576005 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377022bf-2be6-4f99-b9ac-2956c85b275e admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:09.576005 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377022bf-2be6-4f99-b9ac-2956c85b275e admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:09.576005 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377022bf-2be6-4f99-b9ac-2956c85b275e admin admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:09.576005 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377022bf-2be6-4f99-b9ac-2956c85b275e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:09.576700 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377022bf-2be6-4f99-b9ac-2956c85b275e admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:09.597964 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 979/1920] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Sep 23 15:21:09 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:21:09.604842 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99702b8d-cf6f-4b02-8727-0d76bd075b78 admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:09.633594 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99702b8d-cf6f-4b02-8727-0d76bd075b78 admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:09.652701 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae29f519-19c3-41f9-a60c-663cefd632b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:09.654699 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae29f519-19c3-41f9-a60c-663cefd632b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:09.655100 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae29f519-19c3-41f9-a60c-663cefd632b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:09.657242 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae29f519-19c3-41f9-a60c-663cefd632b6 admin admin] PATH_INFO: `/v3/projects/bb7c99deb41246d78733f3e91a015614/users/8ea8ba3e80394fa1b9f5e3b66afe99d4/roles/8e2919e79df040e8926f21607818b58c` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:09.662519 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae29f519-19c3-41f9-a60c-663cefd632b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb7c99deb41246d78733f3e91a015614, user_id=8ea8ba3e80394fa1b9f5e3b66afe99d4, role_id=8e2919e79df040e8926f21607818b58c)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:09.666432 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae29f519-19c3-41f9-a60c-663cefd632b6 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:09.701937 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 942/1921] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Sep 23 15:21:09 2022] PUT /identity/v3/projects/bb7c99deb41246d78733f3e91a015614/users/8ea8ba3e80394fa1b9f5e3b66afe99d4/roles/8e2919e79df040e8926f21607818b58c => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:21:09.714370 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fd2aae-5e13-4a30-8815-2f439d8372f1 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:09.714370 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fd2aae-5e13-4a30-8815-2f439d8372f1 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:09.714909 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fd2aae-5e13-4a30-8815-2f439d8372f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:09.768724 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-93fd2aae-5e13-4a30-8815-2f439d8372f1 None None] MFA Rules not processed for user `8ea8ba3e80394fa1b9f5e3b66afe99d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:21:09.770035 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-93fd2aae-5e13-4a30-8815-2f439d8372f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:09.891277 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 980/1922] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:21:09 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:21:09.909723 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae29f519-19c3-41f9-a60c-663cefd632b6 admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:09.912588 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-ae29f519-19c3-41f9-a60c-663cefd632b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:10.073869 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae29f519-19c3-41f9-a60c-663cefd632b6 admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:10.080264 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6901947f-0432-420d-b4ac-a2110b939e0c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feca6278be904ef8b0dc580a67e477f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:10.081251 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6901947f-0432-420d-b4ac-a2110b939e0c service cinder] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:10.081698 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6901947f-0432-420d-b4ac-a2110b939e0c service cinder] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:10.082189 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6901947f-0432-420d-b4ac-a2110b939e0c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:10.109833 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6901947f-0432-420d-b4ac-a2110b939e0c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:10.114421 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6901947f-0432-420d-b4ac-a2110b939e0c service cinder] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:10.137539 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 943/1923] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:21:09 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:21:10.287360 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fd2aae-5e13-4a30-8815-2f439d8372f1 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:10.288876 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-93fd2aae-5e13-4a30-8815-2f439d8372f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:10.436352 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fd2aae-5e13-4a30-8815-2f439d8372f1 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:10.441830 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e6132d-0ec1-42a3-aab4-fd28f7f69b09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:10.442592 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e6132d-0ec1-42a3-aab4-fd28f7f69b09 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:10.442998 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e6132d-0ec1-42a3-aab4-fd28f7f69b09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:10.443375 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e6132d-0ec1-42a3-aab4-fd28f7f69b09 admin admin] PATH_INFO: `/v3/projects/0cb6baa322f24f6bb24f38a4b921052e` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:10.444735 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e6132d-0ec1-42a3-aab4-fd28f7f69b09 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0cb6baa322f24f6bb24f38a4b921052e)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:10.447807 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e6132d-0ec1-42a3-aab4-fd28f7f69b09 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:10.571874 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:21:10.730171 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-e7e6132d-0ec1-42a3-aab4-fd28f7f69b09 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0cb6baa322f24f6bb24f38a4b921052e', 'request_id': 'req-e7e6132d-0ec1-42a3-aab4-fd28f7f69b09'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:10.740301 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-e7e6132d-0ec1-42a3-aab4-fd28f7f69b09 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0cb6baa322f24f6bb24f38a4b921052e', 'request_id': 'req-e7e6132d-0ec1-42a3-aab4-fd28f7f69b09'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:10.748764 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-e7e6132d-0ec1-42a3-aab4-fd28f7f69b09 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0cb6baa322f24f6bb24f38a4b921052e', 'request_id': 'req-e7e6132d-0ec1-42a3-aab4-fd28f7f69b09'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:10.925853 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-e7e6132d-0ec1-42a3-aab4-fd28f7f69b09 admin admin] The token cache is being invalidate because project 0cb6baa322f24f6bb24f38a4b921052e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81994) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 15:21:10.926206 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-e7e6132d-0ec1-42a3-aab4-fd28f7f69b09 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:10.927079 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-e7e6132d-0ec1-42a3-aab4-fd28f7f69b09 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:10.929688 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 981/1924] 10.222.0.55 () {58 vars in 1233 bytes} [Fri Sep 23 15:21:10 2022] DELETE /identity/v3/projects/0cb6baa322f24f6bb24f38a4b921052e => generated 0 bytes in 643 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:21:10.934727 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379c786c-39b4-4af8-a979-02a560393e48 None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:10.935286 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379c786c-39b4-4af8-a979-02a560393e48 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:10.935490 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379c786c-39b4-4af8-a979-02a560393e48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:10.959753 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-379c786c-39b4-4af8-a979-02a560393e48 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:21:10.961226 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-379c786c-39b4-4af8-a979-02a560393e48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:11.026028 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 944/1925] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:21:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:21:11.035722 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdc8fac-cb0f-4c97-ab00-7b68d1c67c25 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:11.036106 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdc8fac-cb0f-4c97-ab00-7b68d1c67c25 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:11.041137 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdc8fac-cb0f-4c97-ab00-7b68d1c67c25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:11.064097 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-0cdc8fac-cb0f-4c97-ab00-7b68d1c67c25 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:21:11.064097 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-0cdc8fac-cb0f-4c97-ab00-7b68d1c67c25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:11.083243 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379c786c-39b4-4af8-a979-02a560393e48 None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:11.084991 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-379c786c-39b4-4af8-a979-02a560393e48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:11.091862 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 982/1926] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:21:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:21:11.092981 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdc8fac-cb0f-4c97-ab00-7b68d1c67c25 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:11.099483 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379c786c-39b4-4af8-a979-02a560393e48 None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:11.106104 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdc8fac-cb0f-4c97-ab00-7b68d1c67c25 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:11.107102 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d660b7a5-285d-4ee2-a044-22e5bf9e10c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:11.108757 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d660b7a5-285d-4ee2-a044-22e5bf9e10c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:11.109478 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d660b7a5-285d-4ee2-a044-22e5bf9e10c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:11.110515 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d660b7a5-285d-4ee2-a044-22e5bf9e10c0 admin admin] PATH_INFO: `/v3/users/e64768b37cd54dee88b845382a203869` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:11.113059 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97385c23-0e6c-4b45-9ec7-221d2f1285da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:11.113419 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d660b7a5-285d-4ee2-a044-22e5bf9e10c0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e64768b37cd54dee88b845382a203869)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:11.113533 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97385c23-0e6c-4b45-9ec7-221d2f1285da admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:11.114304 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97385c23-0e6c-4b45-9ec7-221d2f1285da admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:11.114304 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97385c23-0e6c-4b45-9ec7-221d2f1285da admin admin] PATH_INFO: `/v3/domains` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:11.115004 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97385c23-0e6c-4b45-9ec7-221d2f1285da admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:21:11.115656 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97385c23-0e6c-4b45-9ec7-221d2f1285da admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:11.117322 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97385c23-0e6c-4b45-9ec7-221d2f1285da admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:11.140527 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d660b7a5-285d-4ee2-a044-22e5bf9e10c0 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:11.140775 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 983/1927] 10.222.0.55 () {58 vars in 1153 bytes} [Fri Sep 23 15:21:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:21:11.140775 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97385c23-0e6c-4b45-9ec7-221d2f1285da admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:11.154077 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97385c23-0e6c-4b45-9ec7-221d2f1285da admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:11.167506 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a52075-5333-469a-8a41-c0bd950b810a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:11.168237 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a52075-5333-469a-8a41-c0bd950b810a admin admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:11.168528 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a52075-5333-469a-8a41-c0bd950b810a admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:11.168889 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a52075-5333-469a-8a41-c0bd950b810a admin admin] PATH_INFO: `/v3/projects` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:11.169539 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a52075-5333-469a-8a41-c0bd950b810a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:11.171463 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a52075-5333-469a-8a41-c0bd950b810a admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:11.353697 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 984/1928] 10.222.0.55 () {60 vars in 1153 bytes} [Fri Sep 23 15:21:11 2022] POST /identity/v3/projects => generated 377 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:21:11.355121 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a52075-5333-469a-8a41-c0bd950b810a admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:11.395997 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-d660b7a5-285d-4ee2-a044-22e5bf9e10c0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e64768b37cd54dee88b845382a203869'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:11.411084 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-d660b7a5-285d-4ee2-a044-22e5bf9e10c0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e64768b37cd54dee88b845382a203869'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:11.419943 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-d660b7a5-285d-4ee2-a044-22e5bf9e10c0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e64768b37cd54dee88b845382a203869'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:11.427090 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a52075-5333-469a-8a41-c0bd950b810a admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:11.434193 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bf6cf2-6cef-45b3-a5df-516ddbbd7b30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:11.434193 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bf6cf2-6cef-45b3-a5df-516ddbbd7b30 admin admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:11.434193 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bf6cf2-6cef-45b3-a5df-516ddbbd7b30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:11.434193 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bf6cf2-6cef-45b3-a5df-516ddbbd7b30 admin admin] PATH_INFO: `/v3/users` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:11.434631 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bf6cf2-6cef-45b3-a5df-516ddbbd7b30 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:11.436414 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bf6cf2-6cef-45b3-a5df-516ddbbd7b30 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:11.635892 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-d660b7a5-285d-4ee2-a044-22e5bf9e10c0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e64768b37cd54dee88b845382a203869'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:11.636678 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-d660b7a5-285d-4ee2-a044-22e5bf9e10c0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e64768b37cd54dee88b845382a203869'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:11.651079 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 945/1929] 10.222.0.55 () {58 vars in 1224 bytes} [Fri Sep 23 15:21:11 2022] DELETE /identity/v3/users/e64768b37cd54dee88b845382a203869 => generated 0 bytes in 561 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:21:11.651348 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 985/1930] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Sep 23 15:21:11 2022] POST /identity/v3/users => generated 289 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 23 15:21:11.654205 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8036b384-3afd-46c7-b674-010089efe4f7 None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:11.654967 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8036b384-3afd-46c7-b674-010089efe4f7 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:11.655130 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8036b384-3afd-46c7-b674-010089efe4f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:11.655810 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bf6cf2-6cef-45b3-a5df-516ddbbd7b30 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:11.657535 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-74bf6cf2-6cef-45b3-a5df-516ddbbd7b30 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:11.670562 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bf6cf2-6cef-45b3-a5df-516ddbbd7b30 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:11.675490 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-8036b384-3afd-46c7-b674-010089efe4f7 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:21:11.676732 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-8036b384-3afd-46c7-b674-010089efe4f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:11.726429 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4559ac-8264-42c0-9991-374896f154d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:11.727049 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4559ac-8264-42c0-9991-374896f154d1 admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:11.727823 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4559ac-8264-42c0-9991-374896f154d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:11.728133 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4559ac-8264-42c0-9991-374896f154d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:11.728780 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4559ac-8264-42c0-9991-374896f154d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:11.730489 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4559ac-8264-42c0-9991-374896f154d1 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:11.738234 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 946/1931] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:21:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 15:21:11.744424 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 986/1932] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Sep 23 15:21:11 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 23 15:21:11.745556 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8036b384-3afd-46c7-b674-010089efe4f7 None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:11.749495 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4559ac-8264-42c0-9991-374896f154d1 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:11.751188 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-7b4559ac-8264-42c0-9991-374896f154d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:11.755185 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8036b384-3afd-46c7-b674-010089efe4f7 None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:11.759905 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44dc30b4-518a-4542-9b5a-4fc81f2c67ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:11.760565 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dc30b4-518a-4542-9b5a-4fc81f2c67ee admin admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:11.760875 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dc30b4-518a-4542-9b5a-4fc81f2c67ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:11.761149 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dc30b4-518a-4542-9b5a-4fc81f2c67ee admin admin] PATH_INFO: `/v3/roles` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:11.761806 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44dc30b4-518a-4542-9b5a-4fc81f2c67ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:11.763638 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44dc30b4-518a-4542-9b5a-4fc81f2c67ee admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:11.777376 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 947/1933] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Sep 23 15:21:11 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 23 15:21:11.779301 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44dc30b4-518a-4542-9b5a-4fc81f2c67ee admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:11.794236 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44dc30b4-518a-4542-9b5a-4fc81f2c67ee admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:11.795381 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa1e9e3-08d9-4aa7-98ba-666609ecaa39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:11.796073 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa1e9e3-08d9-4aa7-98ba-666609ecaa39 admin admin] REQUEST_METHOD: `PUT` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:11.796352 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa1e9e3-08d9-4aa7-98ba-666609ecaa39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:11.796622 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa1e9e3-08d9-4aa7-98ba-666609ecaa39 admin admin] PATH_INFO: `/v3/projects/078e0d3918e749e5aa9c00b1f3ce2114/users/ab6a43229be044669d705bc72edc96f4/roles/af257be2dee54d6b942f58650196582c` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:11.816176 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4559ac-8264-42c0-9991-374896f154d1 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:11.816301 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa1e9e3-08d9-4aa7-98ba-666609ecaa39 admin admin] RBAC: Authorizing `identity:create_grant(project_id=078e0d3918e749e5aa9c00b1f3ce2114, user_id=ab6a43229be044669d705bc72edc96f4, role_id=af257be2dee54d6b942f58650196582c)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:11.818172 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa1e9e3-08d9-4aa7-98ba-666609ecaa39 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:11.820617 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d824e7e5-002f-42be-a0a5-f44a61db4061 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd259178570864ffeb936ee10dfec6b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:11.821628 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d824e7e5-002f-42be-a0a5-f44a61db4061 service neutron] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:11.821914 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d824e7e5-002f-42be-a0a5-f44a61db4061 service neutron] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:11.822192 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d824e7e5-002f-42be-a0a5-f44a61db4061 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:11.832265 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d824e7e5-002f-42be-a0a5-f44a61db4061 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:11.833757 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d824e7e5-002f-42be-a0a5-f44a61db4061 service neutron] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:11.843669 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 987/1934] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:21:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 23 15:21:12.020629 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 948/1935] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Sep 23 15:21:11 2022] PUT /identity/v3/projects/078e0d3918e749e5aa9c00b1f3ce2114/users/ab6a43229be044669d705bc72edc96f4/roles/af257be2dee54d6b942f58650196582c => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:21:12.033141 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c003ae-e9cb-4602-b400-bb5eab99e468 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:12.033618 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c003ae-e9cb-4602-b400-bb5eab99e468 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:12.033987 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c003ae-e9cb-4602-b400-bb5eab99e468 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:12.083576 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-75c003ae-e9cb-4602-b400-bb5eab99e468 None None] MFA Rules not processed for user `ab6a43229be044669d705bc72edc96f4`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:21:12.085629 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-75c003ae-e9cb-4602-b400-bb5eab99e468 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:12.181312 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 988/1936] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:21:12 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:21:12.185838 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa1e9e3-08d9-4aa7-98ba-666609ecaa39 admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:12.252789 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c003ae-e9cb-4602-b400-bb5eab99e468 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:12.254996 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-75c003ae-e9cb-4602-b400-bb5eab99e468 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:12.287016 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa1e9e3-08d9-4aa7-98ba-666609ecaa39 admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:12.292903 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577998e0-70fb-44b9-bb98-57a32ae8a29d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:12.293806 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577998e0-70fb-44b9-bb98-57a32ae8a29d admin admin] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:12.294082 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577998e0-70fb-44b9-bb98-57a32ae8a29d admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:12.294484 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577998e0-70fb-44b9-bb98-57a32ae8a29d admin admin] PATH_INFO: `/v3/projects` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:12.295143 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577998e0-70fb-44b9-bb98-57a32ae8a29d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:12.297101 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577998e0-70fb-44b9-bb98-57a32ae8a29d admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:12.349361 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c003ae-e9cb-4602-b400-bb5eab99e468 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:12.349361 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cb3599-8a57-40a7-8d21-6a400a1acac5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:12.349361 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cb3599-8a57-40a7-8d21-6a400a1acac5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:12.349361 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cb3599-8a57-40a7-8d21-6a400a1acac5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:12.349718 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cb3599-8a57-40a7-8d21-6a400a1acac5 admin admin] PATH_INFO: `/v3/users/faff03e3e5e7483ab8d21aac41447e1e` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:12.349718 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cb3599-8a57-40a7-8d21-6a400a1acac5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=faff03e3e5e7483ab8d21aac41447e1e)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:12.349718 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cb3599-8a57-40a7-8d21-6a400a1acac5 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:12.519140 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 949/1937] 10.222.0.55 () {60 vars in 1153 bytes} [Fri Sep 23 15:21:12 2022] POST /identity/v3/projects => generated 375 bytes in 331 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:21:12.519140 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577998e0-70fb-44b9-bb98-57a32ae8a29d admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:12.520763 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-577998e0-70fb-44b9-bb98-57a32ae8a29d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:12.601491 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577998e0-70fb-44b9-bb98-57a32ae8a29d admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:12.607667 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac647081-fa73-411c-a86e-6729a9b18a22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:12.608351 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac647081-fa73-411c-a86e-6729a9b18a22 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:12.608638 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac647081-fa73-411c-a86e-6729a9b18a22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:12.608941 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac647081-fa73-411c-a86e-6729a9b18a22 admin admin] PATH_INFO: `/v3/projects/b6fdf271fa004fd796efd1eaca24340f` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:12.610064 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac647081-fa73-411c-a86e-6729a9b18a22 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b6fdf271fa004fd796efd1eaca24340f)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:12.612078 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac647081-fa73-411c-a86e-6729a9b18a22 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:12.673751 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-d8cb3599-8a57-40a7-8d21-6a400a1acac5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'faff03e3e5e7483ab8d21aac41447e1e'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:12.691583 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-d8cb3599-8a57-40a7-8d21-6a400a1acac5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'faff03e3e5e7483ab8d21aac41447e1e'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:12.701578 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-d8cb3599-8a57-40a7-8d21-6a400a1acac5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'faff03e3e5e7483ab8d21aac41447e1e'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:12.763842 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-ac647081-fa73-411c-a86e-6729a9b18a22 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b6fdf271fa004fd796efd1eaca24340f', 'request_id': 'req-ac647081-fa73-411c-a86e-6729a9b18a22'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:12.770648 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-d8cb3599-8a57-40a7-8d21-6a400a1acac5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'faff03e3e5e7483ab8d21aac41447e1e'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:12.771752 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-d8cb3599-8a57-40a7-8d21-6a400a1acac5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'faff03e3e5e7483ab8d21aac41447e1e'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:12.772793 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-ac647081-fa73-411c-a86e-6729a9b18a22 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b6fdf271fa004fd796efd1eaca24340f', 'request_id': 'req-ac647081-fa73-411c-a86e-6729a9b18a22'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:12.788930 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 989/1938] 10.222.0.55 () {58 vars in 1224 bytes} [Fri Sep 23 15:21:12 2022] DELETE /identity/v3/users/faff03e3e5e7483ab8d21aac41447e1e => generated 0 bytes in 528 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 15:21:12.788930 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cb3599-8a57-40a7-8d21-6a400a1acac5 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:12.788930 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-d8cb3599-8a57-40a7-8d21-6a400a1acac5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:12.853771 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cb3599-8a57-40a7-8d21-6a400a1acac5 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:12.858578 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a69a778-84ee-4476-adab-d2f690c98562 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:12.859208 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a69a778-84ee-4476-adab-d2f690c98562 admin admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:12.859496 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a69a778-84ee-4476-adab-d2f690c98562 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:12.859820 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a69a778-84ee-4476-adab-d2f690c98562 admin admin] PATH_INFO: `/v3/users` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:12.860540 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a69a778-84ee-4476-adab-d2f690c98562 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:12.862529 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a69a778-84ee-4476-adab-d2f690c98562 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:12.927754 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-ac647081-fa73-411c-a86e-6729a9b18a22 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b6fdf271fa004fd796efd1eaca24340f', 'request_id': 'req-ac647081-fa73-411c-a86e-6729a9b18a22'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:12.947268 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-ac647081-fa73-411c-a86e-6729a9b18a22 admin admin] The token cache is being invalidate because project b6fdf271fa004fd796efd1eaca24340f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81993) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 15:21:12.947573 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-ac647081-fa73-411c-a86e-6729a9b18a22 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:12.948368 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-ac647081-fa73-411c-a86e-6729a9b18a22 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:12.950177 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 950/1939] 10.222.0.55 () {58 vars in 1233 bytes} [Fri Sep 23 15:21:12 2022] DELETE /identity/v3/projects/b6fdf271fa004fd796efd1eaca24340f => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:21:12.953500 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5054f7df-4699-49d6-9f69-e95baead21da None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:12.953788 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5054f7df-4699-49d6-9f69-e95baead21da None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:12.954099 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5054f7df-4699-49d6-9f69-e95baead21da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:12.974677 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-5054f7df-4699-49d6-9f69-e95baead21da None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:21:12.975885 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-5054f7df-4699-49d6-9f69-e95baead21da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:12.986045 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 951/1940] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:21:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:21:12.986045 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5054f7df-4699-49d6-9f69-e95baead21da None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:12.987233 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-5054f7df-4699-49d6-9f69-e95baead21da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:12.999396 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5054f7df-4699-49d6-9f69-e95baead21da None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:13.004197 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5b5fb5-f79c-4b89-8f97-850137c6b5ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:13.011672 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5b5fb5-f79c-4b89-8f97-850137c6b5ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:13.011672 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5b5fb5-f79c-4b89-8f97-850137c6b5ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:13.011672 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5b5fb5-f79c-4b89-8f97-850137c6b5ac admin admin] PATH_INFO: `/v3/users/551f2d4cd3994fd8a44f771d7cfb3cf2` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:13.011672 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5b5fb5-f79c-4b89-8f97-850137c6b5ac admin admin] RBAC: Authorizing `identity:delete_user(user_id=551f2d4cd3994fd8a44f771d7cfb3cf2)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:13.011672 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5b5fb5-f79c-4b89-8f97-850137c6b5ac admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:13.053189 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 990/1941] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Sep 23 15:21:12 2022] POST /identity/v3/users => generated 294 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:21:13.053949 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a69a778-84ee-4476-adab-d2f690c98562 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:13.055680 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-3a69a778-84ee-4476-adab-d2f690c98562 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:13.113263 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a69a778-84ee-4476-adab-d2f690c98562 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:13.117013 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1719815c-ef55-40fe-ada0-b1f34939e996 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd259178570864ffeb936ee10dfec6b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:13.117629 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1719815c-ef55-40fe-ada0-b1f34939e996 service neutron] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:13.118000 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1719815c-ef55-40fe-ada0-b1f34939e996 service neutron] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:13.118280 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1719815c-ef55-40fe-ada0-b1f34939e996 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:13.127779 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1719815c-ef55-40fe-ada0-b1f34939e996 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:13.131844 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1719815c-ef55-40fe-ada0-b1f34939e996 service neutron] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:13.140431 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 991/1942] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:21:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:21:13.141205 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1719815c-ef55-40fe-ada0-b1f34939e996 service neutron] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:13.142562 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-1719815c-ef55-40fe-ada0-b1f34939e996 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:13.156323 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1719815c-ef55-40fe-ada0-b1f34939e996 service neutron] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:13.162859 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97880ad-d9ca-445b-8c68-c2947f111ae2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:13.163497 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97880ad-d9ca-445b-8c68-c2947f111ae2 admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:13.163783 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97880ad-d9ca-445b-8c68-c2947f111ae2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:13.164096 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97880ad-d9ca-445b-8c68-c2947f111ae2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:13.164810 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97880ad-d9ca-445b-8c68-c2947f111ae2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:13.166713 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97880ad-d9ca-445b-8c68-c2947f111ae2 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:13.177908 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 992/1943] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Sep 23 15:21:13 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:21:13.183365 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97880ad-d9ca-445b-8c68-c2947f111ae2 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:13.205186 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97880ad-d9ca-445b-8c68-c2947f111ae2 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:13.212560 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050d1c4f-d439-469d-8a07-ca1bbb004878 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:13.213559 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050d1c4f-d439-469d-8a07-ca1bbb004878 admin admin] REQUEST_METHOD: `PUT` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:13.213683 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050d1c4f-d439-469d-8a07-ca1bbb004878 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:13.214000 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050d1c4f-d439-469d-8a07-ca1bbb004878 admin admin] PATH_INFO: `/v3/projects/33fe28df3f0547da9b76abb87b55902d/users/dac4337d429a4c41a91629e8d5ae1fed/roles/8e2919e79df040e8926f21607818b58c` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:13.230741 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050d1c4f-d439-469d-8a07-ca1bbb004878 admin admin] RBAC: Authorizing `identity:create_grant(project_id=33fe28df3f0547da9b76abb87b55902d, user_id=dac4337d429a4c41a91629e8d5ae1fed, role_id=8e2919e79df040e8926f21607818b58c)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:13.233531 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050d1c4f-d439-469d-8a07-ca1bbb004878 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:13.344050 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-4d5b5fb5-f79c-4b89-8f97-850137c6b5ac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '551f2d4cd3994fd8a44f771d7cfb3cf2'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:13.356741 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-4d5b5fb5-f79c-4b89-8f97-850137c6b5ac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '551f2d4cd3994fd8a44f771d7cfb3cf2'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:13.365281 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-4d5b5fb5-f79c-4b89-8f97-850137c6b5ac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '551f2d4cd3994fd8a44f771d7cfb3cf2'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:13.399865 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 993/1944] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Sep 23 15:21:13 2022] PUT /identity/v3/projects/33fe28df3f0547da9b76abb87b55902d/users/dac4337d429a4c41a91629e8d5ae1fed/roles/8e2919e79df040e8926f21607818b58c => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:21:13.404052 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050d1c4f-d439-469d-8a07-ca1bbb004878 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:13.464608 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-4d5b5fb5-f79c-4b89-8f97-850137c6b5ac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '551f2d4cd3994fd8a44f771d7cfb3cf2'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:13.465415 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-4d5b5fb5-f79c-4b89-8f97-850137c6b5ac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '551f2d4cd3994fd8a44f771d7cfb3cf2'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:13.470349 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050d1c4f-d439-469d-8a07-ca1bbb004878 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:13.474608 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 952/1945] 10.222.0.55 () {58 vars in 1224 bytes} [Fri Sep 23 15:21:12 2022] DELETE /identity/v3/users/551f2d4cd3994fd8a44f771d7cfb3cf2 => generated 0 bytes in 489 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:21:13.476119 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de77052-59bf-40c9-b10f-d35d65e2d835 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:13.476772 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de77052-59bf-40c9-b10f-d35d65e2d835 admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:13.477056 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de77052-59bf-40c9-b10f-d35d65e2d835 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:13.477364 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de77052-59bf-40c9-b10f-d35d65e2d835 admin admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:13.478699 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de77052-59bf-40c9-b10f-d35d65e2d835 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:13.480456 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de77052-59bf-40c9-b10f-d35d65e2d835 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:13.491709 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 994/1946] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Sep 23 15:21:13 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 23 15:21:13.496927 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5b5fb5-f79c-4b89-8f97-850137c6b5ac admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:13.498350 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-4d5b5fb5-f79c-4b89-8f97-850137c6b5ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:13.583959 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5b5fb5-f79c-4b89-8f97-850137c6b5ac admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:13.593172 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c749b820-3e99-4b59-8446-2157904a71ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:13.593937 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c749b820-3e99-4b59-8446-2157904a71ab admin admin] REQUEST_METHOD: `PUT` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:13.594207 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c749b820-3e99-4b59-8446-2157904a71ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:13.594483 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c749b820-3e99-4b59-8446-2157904a71ab admin admin] PATH_INFO: `/v3/projects/33fe28df3f0547da9b76abb87b55902d/users/dac4337d429a4c41a91629e8d5ae1fed/roles/8e2919e79df040e8926f21607818b58c` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:13.597103 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c749b820-3e99-4b59-8446-2157904a71ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=33fe28df3f0547da9b76abb87b55902d, user_id=dac4337d429a4c41a91629e8d5ae1fed, role_id=8e2919e79df040e8926f21607818b58c)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:13.600411 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c749b820-3e99-4b59-8446-2157904a71ab admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:13.611841 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 953/1947] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Sep 23 15:21:13 2022] PUT /identity/v3/projects/33fe28df3f0547da9b76abb87b55902d/users/dac4337d429a4c41a91629e8d5ae1fed/roles/8e2919e79df040e8926f21607818b58c => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:21:13.624376 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dba87a6-d3e2-4d71-a63c-6e8c7077ce97 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:13.624750 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dba87a6-d3e2-4d71-a63c-6e8c7077ce97 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:13.625123 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dba87a6-d3e2-4d71-a63c-6e8c7077ce97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:13.654683 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-7dba87a6-d3e2-4d71-a63c-6e8c7077ce97 None None] MFA Rules not processed for user `dac4337d429a4c41a91629e8d5ae1fed`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:21:13.655897 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-7dba87a6-d3e2-4d71-a63c-6e8c7077ce97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:13.733371 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 995/1948] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:21:13 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 15:21:13.744768 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c749b820-3e99-4b59-8446-2157904a71ab admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:13.744768 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-c749b820-3e99-4b59-8446-2157904a71ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:13.809392 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c749b820-3e99-4b59-8446-2157904a71ab admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:13.814178 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54851985-5bcb-4e40-b49e-2c453e58eac7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feca6278be904ef8b0dc580a67e477f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:13.815285 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54851985-5bcb-4e40-b49e-2c453e58eac7 service cinder] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:13.815285 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54851985-5bcb-4e40-b49e-2c453e58eac7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:13.815503 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54851985-5bcb-4e40-b49e-2c453e58eac7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:13.817476 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-54851985-5bcb-4e40-b49e-2c453e58eac7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:13.870737 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dba87a6-d3e2-4d71-a63c-6e8c7077ce97 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:13.872207 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-7dba87a6-d3e2-4d71-a63c-6e8c7077ce97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:13.902005 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54851985-5bcb-4e40-b49e-2c453e58eac7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:13.906197 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54851985-5bcb-4e40-b49e-2c453e58eac7 service cinder] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:13.922504 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 954/1949] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:21:13 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:21:13.960686 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dba87a6-d3e2-4d71-a63c-6e8c7077ce97 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:13.966732 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d91ed8b-a234-4c29-b908-e9517c2ee931 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:13.967447 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d91ed8b-a234-4c29-b908-e9517c2ee931 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:13.967736 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d91ed8b-a234-4c29-b908-e9517c2ee931 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:13.968066 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d91ed8b-a234-4c29-b908-e9517c2ee931 admin admin] PATH_INFO: `/v3/projects/249110bfa15441cdb278c5b3d85a49a8` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:13.969381 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d91ed8b-a234-4c29-b908-e9517c2ee931 admin admin] RBAC: Authorizing `identity:delete_project(project_id=249110bfa15441cdb278c5b3d85a49a8)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:13.971371 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d91ed8b-a234-4c29-b908-e9517c2ee931 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:14.090103 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54851985-5bcb-4e40-b49e-2c453e58eac7 service cinder] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:14.091695 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-54851985-5bcb-4e40-b49e-2c453e58eac7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:14.110050 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54851985-5bcb-4e40-b49e-2c453e58eac7 service cinder] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:14.121403 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3bebf7-dc02-49b7-8394-c3469faab278 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:14.123562 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3bebf7-dc02-49b7-8394-c3469faab278 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:14.124206 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3bebf7-dc02-49b7-8394-c3469faab278 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:14.124762 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3bebf7-dc02-49b7-8394-c3469faab278 admin admin] PATH_INFO: `/v3/projects/d6b3cf777856438cad979cb178473936` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:14.127058 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3bebf7-dc02-49b7-8394-c3469faab278 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d6b3cf777856438cad979cb178473936)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:14.130408 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3bebf7-dc02-49b7-8394-c3469faab278 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:14.136998 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-5d91ed8b-a234-4c29-b908-e9517c2ee931 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '249110bfa15441cdb278c5b3d85a49a8', 'request_id': 'req-5d91ed8b-a234-4c29-b908-e9517c2ee931'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:14.147892 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-5d91ed8b-a234-4c29-b908-e9517c2ee931 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '249110bfa15441cdb278c5b3d85a49a8', 'request_id': 'req-5d91ed8b-a234-4c29-b908-e9517c2ee931'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:14.157372 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-5d91ed8b-a234-4c29-b908-e9517c2ee931 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '249110bfa15441cdb278c5b3d85a49a8', 'request_id': 'req-5d91ed8b-a234-4c29-b908-e9517c2ee931'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:14.281689 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-5d91ed8b-a234-4c29-b908-e9517c2ee931 admin admin] The token cache is being invalidate because project 249110bfa15441cdb278c5b3d85a49a8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81994) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 15:21:14.282012 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-5d91ed8b-a234-4c29-b908-e9517c2ee931 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:14.282647 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-5d91ed8b-a234-4c29-b908-e9517c2ee931 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:14.284299 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 996/1950] 10.222.0.55 () {58 vars in 1233 bytes} [Fri Sep 23 15:21:13 2022] DELETE /identity/v3/projects/249110bfa15441cdb278c5b3d85a49a8 => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 15:21:14.288105 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d91ed8b-a234-4c29-b908-e9517c2ee931 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:14.289405 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-5d91ed8b-a234-4c29-b908-e9517c2ee931 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:14.322722 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:21:14.346017 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-1a3bebf7-dc02-49b7-8394-c3469faab278 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd6b3cf777856438cad979cb178473936', 'request_id': 'req-1a3bebf7-dc02-49b7-8394-c3469faab278'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:14.351674 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-1a3bebf7-dc02-49b7-8394-c3469faab278 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd6b3cf777856438cad979cb178473936', 'request_id': 'req-1a3bebf7-dc02-49b7-8394-c3469faab278'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:14.357700 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d91ed8b-a234-4c29-b908-e9517c2ee931 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:14.413916 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363a2672-1b18-4399-8830-a3761ef05334 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:14.415434 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363a2672-1b18-4399-8830-a3761ef05334 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:14.415716 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363a2672-1b18-4399-8830-a3761ef05334 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:14.416115 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363a2672-1b18-4399-8830-a3761ef05334 admin admin] PATH_INFO: `/v3/users/8ea8ba3e80394fa1b9f5e3b66afe99d4` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:14.417503 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363a2672-1b18-4399-8830-a3761ef05334 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8ea8ba3e80394fa1b9f5e3b66afe99d4)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:14.419398 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363a2672-1b18-4399-8830-a3761ef05334 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:14.475471 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-1a3bebf7-dc02-49b7-8394-c3469faab278 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd6b3cf777856438cad979cb178473936', 'request_id': 'req-1a3bebf7-dc02-49b7-8394-c3469faab278'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:14.488329 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-1a3bebf7-dc02-49b7-8394-c3469faab278 admin admin] The token cache is being invalidate because project d6b3cf777856438cad979cb178473936 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81993) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 15:21:14.488623 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-1a3bebf7-dc02-49b7-8394-c3469faab278 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:14.489444 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-1a3bebf7-dc02-49b7-8394-c3469faab278 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:14.490940 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 955/1951] 10.222.0.55 () {58 vars in 1233 bytes} [Fri Sep 23 15:21:14 2022] DELETE /identity/v3/projects/d6b3cf777856438cad979cb178473936 => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 15:21:14.495757 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57836540-8f66-4c35-a0c5-d34d6a714245 None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:14.496075 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57836540-8f66-4c35-a0c5-d34d6a714245 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:14.496355 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57836540-8f66-4c35-a0c5-d34d6a714245 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:14.516469 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-57836540-8f66-4c35-a0c5-d34d6a714245 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:21:14.517677 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-57836540-8f66-4c35-a0c5-d34d6a714245 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:14.526192 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 956/1952] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:21:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 15:21:14.533754 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c70c8c-af6b-4c45-8eb6-6e452222744b None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:14.534047 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c70c8c-af6b-4c45-8eb6-6e452222744b None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:14.536597 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c70c8c-af6b-4c45-8eb6-6e452222744b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:14.553992 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-78c70c8c-af6b-4c45-8eb6-6e452222744b None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:21:14.555207 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-78c70c8c-af6b-4c45-8eb6-6e452222744b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:14.563304 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 957/1953] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:21:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 15:21:14.567475 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c70c8c-af6b-4c45-8eb6-6e452222744b None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:14.577172 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c70c8c-af6b-4c45-8eb6-6e452222744b None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:14.581729 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5b82e2-f353-43fb-97a6-9a034ff41022 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:14.582374 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5b82e2-f353-43fb-97a6-9a034ff41022 admin admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:14.582659 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5b82e2-f353-43fb-97a6-9a034ff41022 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:14.582931 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5b82e2-f353-43fb-97a6-9a034ff41022 admin admin] PATH_INFO: `/v3/domains` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:14.583564 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5b82e2-f353-43fb-97a6-9a034ff41022 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:21:14.583867 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5b82e2-f353-43fb-97a6-9a034ff41022 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:14.585889 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5b82e2-f353-43fb-97a6-9a034ff41022 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:14.598128 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 958/1954] 10.222.0.55 () {58 vars in 1153 bytes} [Fri Sep 23 15:21:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 15:21:14.602209 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5b82e2-f353-43fb-97a6-9a034ff41022 admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:14.611541 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5b82e2-f353-43fb-97a6-9a034ff41022 admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:14.616479 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611f3d97-b3eb-4b3d-8e6f-26c7ea2d1046 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:14.618037 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611f3d97-b3eb-4b3d-8e6f-26c7ea2d1046 admin admin] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:14.618330 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611f3d97-b3eb-4b3d-8e6f-26c7ea2d1046 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:14.618611 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611f3d97-b3eb-4b3d-8e6f-26c7ea2d1046 admin admin] PATH_INFO: `/v3/projects` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:14.619481 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611f3d97-b3eb-4b3d-8e6f-26c7ea2d1046 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:14.621609 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611f3d97-b3eb-4b3d-8e6f-26c7ea2d1046 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:14.757034 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-363a2672-1b18-4399-8830-a3761ef05334 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8ea8ba3e80394fa1b9f5e3b66afe99d4'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:14.768687 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:21:14.777093 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-363a2672-1b18-4399-8830-a3761ef05334 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8ea8ba3e80394fa1b9f5e3b66afe99d4'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:14.779628 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 959/1955] 10.222.0.55 () {60 vars in 1153 bytes} [Fri Sep 23 15:21:14 2022] POST /identity/v3/projects => generated 365 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 23 15:21:14.780924 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611f3d97-b3eb-4b3d-8e6f-26c7ea2d1046 admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:14.786655 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-363a2672-1b18-4399-8830-a3761ef05334 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8ea8ba3e80394fa1b9f5e3b66afe99d4'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:14.855005 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611f3d97-b3eb-4b3d-8e6f-26c7ea2d1046 admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:14.862816 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5e9e1e-4ce8-4ce6-8d78-82e545b93c50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:14.862816 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5e9e1e-4ce8-4ce6-8d78-82e545b93c50 admin admin] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:14.862816 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5e9e1e-4ce8-4ce6-8d78-82e545b93c50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:14.862816 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5e9e1e-4ce8-4ce6-8d78-82e545b93c50 admin admin] PATH_INFO: `/v3/users` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:14.863242 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5e9e1e-4ce8-4ce6-8d78-82e545b93c50 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:14.864420 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5e9e1e-4ce8-4ce6-8d78-82e545b93c50 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:14.926802 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-363a2672-1b18-4399-8830-a3761ef05334 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8ea8ba3e80394fa1b9f5e3b66afe99d4'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:14.927510 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-363a2672-1b18-4399-8830-a3761ef05334 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8ea8ba3e80394fa1b9f5e3b66afe99d4'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:14.935098 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 997/1956] 10.222.0.55 () {58 vars in 1224 bytes} [Fri Sep 23 15:21:14 2022] DELETE /identity/v3/users/8ea8ba3e80394fa1b9f5e3b66afe99d4 => generated 0 bytes in 647 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 15:21:15.222976 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 960/1957] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Sep 23 15:21:14 2022] POST /identity/v3/users => generated 283 bytes in 440 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:21:15.224386 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363a2672-1b18-4399-8830-a3761ef05334 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:15.225866 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-363a2672-1b18-4399-8830-a3761ef05334 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:15.323786 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363a2672-1b18-4399-8830-a3761ef05334 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:15.332660 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5e3fd8-7ebe-4900-8709-231d5c507280 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:15.333300 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5e3fd8-7ebe-4900-8709-231d5c507280 admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:15.333603 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5e3fd8-7ebe-4900-8709-231d5c507280 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:15.334334 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5e3fd8-7ebe-4900-8709-231d5c507280 admin admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:15.334534 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5e3fd8-7ebe-4900-8709-231d5c507280 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:15.336195 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5e3fd8-7ebe-4900-8709-231d5c507280 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:15.348816 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 998/1958] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Sep 23 15:21:15 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:21:15.352567 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5e9e1e-4ce8-4ce6-8d78-82e545b93c50 admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:15.364320 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5e9e1e-4ce8-4ce6-8d78-82e545b93c50 admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:15.369950 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb43298-048b-4005-a24a-4dc941d365db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:15.370682 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb43298-048b-4005-a24a-4dc941d365db admin admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:15.370965 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb43298-048b-4005-a24a-4dc941d365db admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:15.371244 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb43298-048b-4005-a24a-4dc941d365db admin admin] PATH_INFO: `/v3/roles` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:15.371927 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb43298-048b-4005-a24a-4dc941d365db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:15.373722 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb43298-048b-4005-a24a-4dc941d365db admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:15.387167 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 961/1959] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Sep 23 15:21:15 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 23 15:21:15.392150 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5e3fd8-7ebe-4900-8709-231d5c507280 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:15.406129 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5e3fd8-7ebe-4900-8709-231d5c507280 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:15.411228 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065fd893-ffa9-4aba-b2ef-f917d5edae34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:15.412130 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065fd893-ffa9-4aba-b2ef-f917d5edae34 admin admin] REQUEST_METHOD: `PUT` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:15.412571 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065fd893-ffa9-4aba-b2ef-f917d5edae34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:15.412852 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065fd893-ffa9-4aba-b2ef-f917d5edae34 admin admin] PATH_INFO: `/v3/projects/0d437d9063644d7b97ea5cea377dc024/users/64d0dfbc6ec64c44a5ac8b652f25e521/roles/af257be2dee54d6b942f58650196582c` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:15.432282 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065fd893-ffa9-4aba-b2ef-f917d5edae34 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d437d9063644d7b97ea5cea377dc024, user_id=64d0dfbc6ec64c44a5ac8b652f25e521, role_id=af257be2dee54d6b942f58650196582c)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:15.435098 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065fd893-ffa9-4aba-b2ef-f917d5edae34 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:15.527276 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb43298-048b-4005-a24a-4dc941d365db admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:15.529351 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-deb43298-048b-4005-a24a-4dc941d365db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:15.543513 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb43298-048b-4005-a24a-4dc941d365db admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:15.548405 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d872f8e5-2505-45ad-902f-0aa926e0c245 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:15.549053 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d872f8e5-2505-45ad-902f-0aa926e0c245 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:15.549333 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d872f8e5-2505-45ad-902f-0aa926e0c245 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:15.549609 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d872f8e5-2505-45ad-902f-0aa926e0c245 admin admin] PATH_INFO: `/v3/projects/bb7c99deb41246d78733f3e91a015614` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:15.550740 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d872f8e5-2505-45ad-902f-0aa926e0c245 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bb7c99deb41246d78733f3e91a015614)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:15.553011 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d872f8e5-2505-45ad-902f-0aa926e0c245 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:15.576715 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 999/1960] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Sep 23 15:21:15 2022] PUT /identity/v3/projects/0d437d9063644d7b97ea5cea377dc024/users/64d0dfbc6ec64c44a5ac8b652f25e521/roles/af257be2dee54d6b942f58650196582c => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:21:15.589086 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c1c289-e829-4ace-901e-496d7dc42148 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:15.589377 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c1c289-e829-4ace-901e-496d7dc42148 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:15.589655 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c1c289-e829-4ace-901e-496d7dc42148 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:15.619397 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-b1c1c289-e829-4ace-901e-496d7dc42148 None None] MFA Rules not processed for user `64d0dfbc6ec64c44a5ac8b652f25e521`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:21:15.620710 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-b1c1c289-e829-4ace-901e-496d7dc42148 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:15.689415 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1000/1961] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:21:15 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 15:21:15.693886 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c1c289-e829-4ace-901e-496d7dc42148 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:15.730827 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-d872f8e5-2505-45ad-902f-0aa926e0c245 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bb7c99deb41246d78733f3e91a015614', 'request_id': 'req-d872f8e5-2505-45ad-902f-0aa926e0c245'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:15.748962 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-d872f8e5-2505-45ad-902f-0aa926e0c245 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bb7c99deb41246d78733f3e91a015614', 'request_id': 'req-d872f8e5-2505-45ad-902f-0aa926e0c245'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:15.760280 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c1c289-e829-4ace-901e-496d7dc42148 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:15.845940 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18bdc4c-a050-48bd-a5a3-5c1fdff9c02a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:15.845940 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18bdc4c-a050-48bd-a5a3-5c1fdff9c02a admin admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:15.845940 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18bdc4c-a050-48bd-a5a3-5c1fdff9c02a admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:15.845940 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18bdc4c-a050-48bd-a5a3-5c1fdff9c02a admin admin] PATH_INFO: `/v3/projects` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:15.846324 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18bdc4c-a050-48bd-a5a3-5c1fdff9c02a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:15.846324 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18bdc4c-a050-48bd-a5a3-5c1fdff9c02a admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:15.879018 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-d872f8e5-2505-45ad-902f-0aa926e0c245 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bb7c99deb41246d78733f3e91a015614', 'request_id': 'req-d872f8e5-2505-45ad-902f-0aa926e0c245'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:15.895573 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-d872f8e5-2505-45ad-902f-0aa926e0c245 admin admin] The token cache is being invalidate because project bb7c99deb41246d78733f3e91a015614 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81993) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 15:21:15.895888 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-d872f8e5-2505-45ad-902f-0aa926e0c245 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:15.896603 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-d872f8e5-2505-45ad-902f-0aa926e0c245 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:15.900664 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 962/1962] 10.222.0.55 () {58 vars in 1233 bytes} [Fri Sep 23 15:21:15 2022] DELETE /identity/v3/projects/bb7c99deb41246d78733f3e91a015614 => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 15:21:15.913563 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1802f86-365e-4c0f-a73b-698d30683b42 None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:15.913890 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1802f86-365e-4c0f-a73b-698d30683b42 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:15.914244 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1802f86-365e-4c0f-a73b-698d30683b42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:15.938599 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-d1802f86-365e-4c0f-a73b-698d30683b42 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:21:15.938599 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-d1802f86-365e-4c0f-a73b-698d30683b42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:15.949115 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 963/1963] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:21:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 15:21:15.956459 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c91df9-1f56-4156-8ff3-b4483f83bbdb None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:15.956883 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c91df9-1f56-4156-8ff3-b4483f83bbdb None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:15.957273 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c91df9-1f56-4156-8ff3-b4483f83bbdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:15.979203 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1001/1964] 10.222.0.55 () {60 vars in 1153 bytes} [Fri Sep 23 15:21:15 2022] POST /identity/v3/projects => generated 365 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 23 15:21:15.980737 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18bdc4c-a050-48bd-a5a3-5c1fdff9c02a admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:15.985757 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-b18bdc4c-a050-48bd-a5a3-5c1fdff9c02a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:15.986491 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-14c91df9-1f56-4156-8ff3-b4483f83bbdb None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:21:15.988701 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-14c91df9-1f56-4156-8ff3-b4483f83bbdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:16.082360 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 964/1965] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:21:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:21:16.082470 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18bdc4c-a050-48bd-a5a3-5c1fdff9c02a admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:16.083870 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c91df9-1f56-4156-8ff3-b4483f83bbdb None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:16.088496 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5febbfd7-05b1-4b55-9427-46f05662f483 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:16.089643 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5febbfd7-05b1-4b55-9427-46f05662f483 admin admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:16.090612 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5febbfd7-05b1-4b55-9427-46f05662f483 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:16.091077 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5febbfd7-05b1-4b55-9427-46f05662f483 admin admin] PATH_INFO: `/v3/users` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:16.092095 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5febbfd7-05b1-4b55-9427-46f05662f483 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:16.093993 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5febbfd7-05b1-4b55-9427-46f05662f483 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:16.094146 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c91df9-1f56-4156-8ff3-b4483f83bbdb None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:16.098967 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a4b22f-4a49-401b-ab8f-d034b7d44a28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:16.099589 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a4b22f-4a49-401b-ab8f-d034b7d44a28 admin admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:16.100212 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a4b22f-4a49-401b-ab8f-d034b7d44a28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:16.100321 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a4b22f-4a49-401b-ab8f-d034b7d44a28 admin admin] PATH_INFO: `/v3/domains` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:16.100909 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a4b22f-4a49-401b-ab8f-d034b7d44a28 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:21:16.101218 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a4b22f-4a49-401b-ab8f-d034b7d44a28 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:16.103387 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a4b22f-4a49-401b-ab8f-d034b7d44a28 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:16.116086 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 965/1966] 10.222.0.55 () {58 vars in 1153 bytes} [Fri Sep 23 15:21:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:21:16.121057 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a4b22f-4a49-401b-ab8f-d034b7d44a28 admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:16.130899 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a4b22f-4a49-401b-ab8f-d034b7d44a28 admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:16.135653 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8786b7-c3f4-4dfa-a93a-2d185da4f5d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:16.136310 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8786b7-c3f4-4dfa-a93a-2d185da4f5d9 admin admin] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:16.136604 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8786b7-c3f4-4dfa-a93a-2d185da4f5d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:16.136888 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8786b7-c3f4-4dfa-a93a-2d185da4f5d9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:16.137516 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8786b7-c3f4-4dfa-a93a-2d185da4f5d9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:16.139718 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8786b7-c3f4-4dfa-a93a-2d185da4f5d9 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:16.187018 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1002/1967] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Sep 23 15:21:15 2022] POST /identity/v3/users => generated 289 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:21:16.188187 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5febbfd7-05b1-4b55-9427-46f05662f483 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:16.198792 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5febbfd7-05b1-4b55-9427-46f05662f483 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:16.203769 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1d0b7b-55e9-426b-ae12-0ea363a53ef0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:16.204415 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1d0b7b-55e9-426b-ae12-0ea363a53ef0 admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:16.204702 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1d0b7b-55e9-426b-ae12-0ea363a53ef0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:16.204980 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1d0b7b-55e9-426b-ae12-0ea363a53ef0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:16.205681 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1d0b7b-55e9-426b-ae12-0ea363a53ef0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:16.207275 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1d0b7b-55e9-426b-ae12-0ea363a53ef0 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:16.218489 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1003/1968] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Sep 23 15:21:16 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 23 15:21:16.228584 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1d0b7b-55e9-426b-ae12-0ea363a53ef0 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:16.233791 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1d0b7b-55e9-426b-ae12-0ea363a53ef0 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:16.239908 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49c6319-c956-42d3-90dd-1d538f7b7623 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:16.239908 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49c6319-c956-42d3-90dd-1d538f7b7623 admin admin] REQUEST_METHOD: `PUT` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:16.239908 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49c6319-c956-42d3-90dd-1d538f7b7623 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:16.239908 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49c6319-c956-42d3-90dd-1d538f7b7623 admin admin] PATH_INFO: `/v3/projects/8eda34d3f24a429ca2627b289a2935d2/users/1542fa130b944a67b4ea2fefb7aa48fe/roles/8e2919e79df040e8926f21607818b58c` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:16.255444 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49c6319-c956-42d3-90dd-1d538f7b7623 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8eda34d3f24a429ca2627b289a2935d2, user_id=1542fa130b944a67b4ea2fefb7aa48fe, role_id=8e2919e79df040e8926f21607818b58c)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:16.258099 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49c6319-c956-42d3-90dd-1d538f7b7623 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:16.285500 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 966/1969] 10.222.0.55 () {60 vars in 1153 bytes} [Fri Sep 23 15:21:16 2022] POST /identity/v3/projects => generated 385 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 23 15:21:16.286513 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8786b7-c3f4-4dfa-a93a-2d185da4f5d9 admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:16.362188 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8786b7-c3f4-4dfa-a93a-2d185da4f5d9 admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:16.365410 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1004/1970] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Sep 23 15:21:16 2022] PUT /identity/v3/projects/8eda34d3f24a429ca2627b289a2935d2/users/1542fa130b944a67b4ea2fefb7aa48fe/roles/8e2919e79df040e8926f21607818b58c => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 15:21:16.370759 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49c6319-c956-42d3-90dd-1d538f7b7623 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:16.451268 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cd3e43-4b51-4727-831f-519be42cefe9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:16.452125 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cd3e43-4b51-4727-831f-519be42cefe9 admin admin] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:16.452390 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cd3e43-4b51-4727-831f-519be42cefe9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:16.452740 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cd3e43-4b51-4727-831f-519be42cefe9 admin admin] PATH_INFO: `/v3/users` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:16.453386 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cd3e43-4b51-4727-831f-519be42cefe9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:16.455321 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cd3e43-4b51-4727-831f-519be42cefe9 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:16.500971 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49c6319-c956-42d3-90dd-1d538f7b7623 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:16.505630 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31921e4d-ae63-4d00-9b46-eab45dd4f8a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:16.506242 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31921e4d-ae63-4d00-9b46-eab45dd4f8a1 admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:16.506524 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31921e4d-ae63-4d00-9b46-eab45dd4f8a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:16.506796 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31921e4d-ae63-4d00-9b46-eab45dd4f8a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:16.507434 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31921e4d-ae63-4d00-9b46-eab45dd4f8a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:16.509197 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31921e4d-ae63-4d00-9b46-eab45dd4f8a1 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:16.520065 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1005/1971] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Sep 23 15:21:16 2022] GET /identity/v3/roles => generated 1420 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 23 15:21:16.527171 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31921e4d-ae63-4d00-9b46-eab45dd4f8a1 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:16.546110 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31921e4d-ae63-4d00-9b46-eab45dd4f8a1 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:16.551271 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556dbf9a-74c0-46b3-b3f6-9b54c5d0c6d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:16.551937 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556dbf9a-74c0-46b3-b3f6-9b54c5d0c6d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:16.552248 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556dbf9a-74c0-46b3-b3f6-9b54c5d0c6d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:16.552542 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556dbf9a-74c0-46b3-b3f6-9b54c5d0c6d2 admin admin] PATH_INFO: `/v3/projects/8eda34d3f24a429ca2627b289a2935d2/users/1542fa130b944a67b4ea2fefb7aa48fe/roles/8e2919e79df040e8926f21607818b58c` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:16.554659 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556dbf9a-74c0-46b3-b3f6-9b54c5d0c6d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8eda34d3f24a429ca2627b289a2935d2, user_id=1542fa130b944a67b4ea2fefb7aa48fe, role_id=8e2919e79df040e8926f21607818b58c)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:16.557589 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556dbf9a-74c0-46b3-b3f6-9b54c5d0c6d2 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:16.568463 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1006/1972] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Sep 23 15:21:16 2022] PUT /identity/v3/projects/8eda34d3f24a429ca2627b289a2935d2/users/1542fa130b944a67b4ea2fefb7aa48fe/roles/8e2919e79df040e8926f21607818b58c => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:21:16.578721 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92a4127-3248-4b03-a0e1-811649769c4c None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:16.579011 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92a4127-3248-4b03-a0e1-811649769c4c None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:16.579282 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92a4127-3248-4b03-a0e1-811649769c4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:16.609016 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-d92a4127-3248-4b03-a0e1-811649769c4c None None] MFA Rules not processed for user `1542fa130b944a67b4ea2fefb7aa48fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:21:16.610261 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-d92a4127-3248-4b03-a0e1-811649769c4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:16.643643 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 967/1973] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Sep 23 15:21:16 2022] POST /identity/v3/users => generated 293 bytes in 354 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:21:16.644507 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cd3e43-4b51-4727-831f-519be42cefe9 admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:16.664126 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cd3e43-4b51-4727-831f-519be42cefe9 admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:16.701479 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1007/1974] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:21:16 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 15:21:16.714712 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92a4127-3248-4b03-a0e1-811649769c4c None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:16.716398 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-d92a4127-3248-4b03-a0e1-811649769c4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:16.732341 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2263446e-bd0d-41a0-bef2-033bc2dcb625 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:16.733068 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2263446e-bd0d-41a0-bef2-033bc2dcb625 admin admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:16.733435 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2263446e-bd0d-41a0-bef2-033bc2dcb625 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:16.733892 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2263446e-bd0d-41a0-bef2-033bc2dcb625 admin admin] PATH_INFO: `/v3/roles` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:16.734548 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2263446e-bd0d-41a0-bef2-033bc2dcb625 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:16.736280 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2263446e-bd0d-41a0-bef2-033bc2dcb625 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:16.763476 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 968/1975] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Sep 23 15:21:16 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 23 15:21:16.766233 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2263446e-bd0d-41a0-bef2-033bc2dcb625 admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:16.776955 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2263446e-bd0d-41a0-bef2-033bc2dcb625 admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:16.781590 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9ec3d5-c077-456f-8e4e-c5a069fa0bb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:16.782194 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9ec3d5-c077-456f-8e4e-c5a069fa0bb9 admin admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:16.782554 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9ec3d5-c077-456f-8e4e-c5a069fa0bb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:16.782826 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9ec3d5-c077-456f-8e4e-c5a069fa0bb9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:16.783461 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9ec3d5-c077-456f-8e4e-c5a069fa0bb9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:16.784142 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92a4127-3248-4b03-a0e1-811649769c4c None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:16.785072 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9ec3d5-c077-456f-8e4e-c5a069fa0bb9 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:16.787832 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc97856c-c9d7-4057-90a3-ed82f5e527ec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feca6278be904ef8b0dc580a67e477f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:16.789005 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc97856c-c9d7-4057-90a3-ed82f5e527ec service cinder] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:16.789289 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc97856c-c9d7-4057-90a3-ed82f5e527ec service cinder] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:16.790094 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc97856c-c9d7-4057-90a3-ed82f5e527ec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:16.802029 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 969/1976] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Sep 23 15:21:16 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 23 15:21:16.802210 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc97856c-c9d7-4057-90a3-ed82f5e527ec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:16.802467 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc97856c-c9d7-4057-90a3-ed82f5e527ec service cinder] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:16.803152 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9ec3d5-c077-456f-8e4e-c5a069fa0bb9 admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:16.819198 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1008/1977] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:21:16 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 23 15:21:16.819295 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9ec3d5-c077-456f-8e4e-c5a069fa0bb9 admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:16.819716 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5daf9f70-2637-4267-90a6-ec1f1297bae8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:16.820423 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5daf9f70-2637-4267-90a6-ec1f1297bae8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:16.820878 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5daf9f70-2637-4267-90a6-ec1f1297bae8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:16.821250 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5daf9f70-2637-4267-90a6-ec1f1297bae8 admin admin] PATH_INFO: `/v3/projects/0b643cc7818c49fb88ac37d962844efb/users/f5a74ea715584ae89d1c72dd5275b466/roles/af257be2dee54d6b942f58650196582c` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:16.838270 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5daf9f70-2637-4267-90a6-ec1f1297bae8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b643cc7818c49fb88ac37d962844efb, user_id=f5a74ea715584ae89d1c72dd5275b466, role_id=af257be2dee54d6b942f58650196582c)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:16.840942 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5daf9f70-2637-4267-90a6-ec1f1297bae8 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:16.965517 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 970/1978] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Sep 23 15:21:16 2022] PUT /identity/v3/projects/0b643cc7818c49fb88ac37d962844efb/users/f5a74ea715584ae89d1c72dd5275b466/roles/af257be2dee54d6b942f58650196582c => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:21:16.980864 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10ac4e6-f079-40bf-b316-7cf7e2af4661 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:16.981105 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10ac4e6-f079-40bf-b316-7cf7e2af4661 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:16.981386 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10ac4e6-f079-40bf-b316-7cf7e2af4661 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:17.013428 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-a10ac4e6-f079-40bf-b316-7cf7e2af4661 None None] MFA Rules not processed for user `f5a74ea715584ae89d1c72dd5275b466`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:21:17.014962 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-a10ac4e6-f079-40bf-b316-7cf7e2af4661 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:17.099723 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1009/1979] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:21:16 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:21:17.139835 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5daf9f70-2637-4267-90a6-ec1f1297bae8 admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:17.203042 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5daf9f70-2637-4267-90a6-ec1f1297bae8 admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:17.206870 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ba7c2f-7078-4f1e-975a-088e39bd1c72 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feca6278be904ef8b0dc580a67e477f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:17.207530 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ba7c2f-7078-4f1e-975a-088e39bd1c72 service cinder] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:17.207833 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ba7c2f-7078-4f1e-975a-088e39bd1c72 service cinder] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:17.208165 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ba7c2f-7078-4f1e-975a-088e39bd1c72 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:17.221975 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ba7c2f-7078-4f1e-975a-088e39bd1c72 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:17.221975 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ba7c2f-7078-4f1e-975a-088e39bd1c72 service cinder] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:17.231813 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 971/1980] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:21:17 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:21:17.252269 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10ac4e6-f079-40bf-b316-7cf7e2af4661 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:17.318573 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10ac4e6-f079-40bf-b316-7cf7e2af4661 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:17.326103 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52514f3c-2abe-4532-af59-5183c641c36e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:17.326744 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52514f3c-2abe-4532-af59-5183c641c36e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:17.327025 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52514f3c-2abe-4532-af59-5183c641c36e admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:17.327305 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52514f3c-2abe-4532-af59-5183c641c36e admin admin] PATH_INFO: `/v3/users/f5a74ea715584ae89d1c72dd5275b466` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:17.328462 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52514f3c-2abe-4532-af59-5183c641c36e admin admin] RBAC: Authorizing `identity:delete_user(user_id=f5a74ea715584ae89d1c72dd5275b466)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:17.330381 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52514f3c-2abe-4532-af59-5183c641c36e admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:17.578607 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-52514f3c-2abe-4532-af59-5183c641c36e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f5a74ea715584ae89d1c72dd5275b466'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:17.592305 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-52514f3c-2abe-4532-af59-5183c641c36e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f5a74ea715584ae89d1c72dd5275b466'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:17.599182 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-52514f3c-2abe-4532-af59-5183c641c36e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f5a74ea715584ae89d1c72dd5275b466'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:17.766823 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-52514f3c-2abe-4532-af59-5183c641c36e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f5a74ea715584ae89d1c72dd5275b466'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:17.767405 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-52514f3c-2abe-4532-af59-5183c641c36e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f5a74ea715584ae89d1c72dd5275b466'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:17.776185 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1010/1981] 10.222.0.55 () {58 vars in 1224 bytes} [Fri Sep 23 15:21:17 2022] DELETE /identity/v3/users/f5a74ea715584ae89d1c72dd5275b466 => generated 0 bytes in 523 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:21:17.779715 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395627e0-7fbd-4526-9f14-3df0d0b7e773 None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:17.780208 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395627e0-7fbd-4526-9f14-3df0d0b7e773 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:17.780574 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395627e0-7fbd-4526-9f14-3df0d0b7e773 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:17.802189 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-395627e0-7fbd-4526-9f14-3df0d0b7e773 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:21:17.803344 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-395627e0-7fbd-4526-9f14-3df0d0b7e773 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:17.873029 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 972/1982] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:21:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:21:17.885301 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52514f3c-2abe-4532-af59-5183c641c36e admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:17.886944 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-52514f3c-2abe-4532-af59-5183c641c36e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:17.956103 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52514f3c-2abe-4532-af59-5183c641c36e admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:17.959717 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8881ec67-6eb0-49e2-95f8-8add577e8965 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd259178570864ffeb936ee10dfec6b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:17.960361 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8881ec67-6eb0-49e2-95f8-8add577e8965 service neutron] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:17.960643 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8881ec67-6eb0-49e2-95f8-8add577e8965 service neutron] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:17.960942 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8881ec67-6eb0-49e2-95f8-8add577e8965 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:17.970342 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8881ec67-6eb0-49e2-95f8-8add577e8965 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:17.971760 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8881ec67-6eb0-49e2-95f8-8add577e8965 service neutron] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:17.982402 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1011/1983] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:21:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:21:18.150391 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395627e0-7fbd-4526-9f14-3df0d0b7e773 None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:18.157409 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-395627e0-7fbd-4526-9f14-3df0d0b7e773 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:18.164081 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395627e0-7fbd-4526-9f14-3df0d0b7e773 None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:18.169087 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1790239b-6c7d-48bd-ad8f-312aa07037d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:18.169716 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1790239b-6c7d-48bd-ad8f-312aa07037d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:18.169997 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1790239b-6c7d-48bd-ad8f-312aa07037d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:18.170270 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1790239b-6c7d-48bd-ad8f-312aa07037d2 admin admin] PATH_INFO: `/v3/users/64d0dfbc6ec64c44a5ac8b652f25e521` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:18.171389 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1790239b-6c7d-48bd-ad8f-312aa07037d2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=64d0dfbc6ec64c44a5ac8b652f25e521)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:18.173384 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1790239b-6c7d-48bd-ad8f-312aa07037d2 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:18.186787 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:21:18.478614 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-1790239b-6c7d-48bd-ad8f-312aa07037d2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '64d0dfbc6ec64c44a5ac8b652f25e521'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:18.490524 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-1790239b-6c7d-48bd-ad8f-312aa07037d2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '64d0dfbc6ec64c44a5ac8b652f25e521'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:18.496379 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:21:18.498956 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-1790239b-6c7d-48bd-ad8f-312aa07037d2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '64d0dfbc6ec64c44a5ac8b652f25e521'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:18.502962 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8881ec67-6eb0-49e2-95f8-8add577e8965 service neutron] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:18.504657 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-8881ec67-6eb0-49e2-95f8-8add577e8965 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:18.520157 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8881ec67-6eb0-49e2-95f8-8add577e8965 service neutron] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:18.525661 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3430717-88ca-4ef1-af14-38e862718d1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:18.526443 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3430717-88ca-4ef1-af14-38e862718d1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:18.526727 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3430717-88ca-4ef1-af14-38e862718d1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:18.526998 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3430717-88ca-4ef1-af14-38e862718d1b admin admin] PATH_INFO: `/v3/projects/0b643cc7818c49fb88ac37d962844efb` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:18.528297 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3430717-88ca-4ef1-af14-38e862718d1b admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b643cc7818c49fb88ac37d962844efb)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:18.530153 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3430717-88ca-4ef1-af14-38e862718d1b admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:18.652172 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-1790239b-6c7d-48bd-ad8f-312aa07037d2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '64d0dfbc6ec64c44a5ac8b652f25e521'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:18.652740 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-1790239b-6c7d-48bd-ad8f-312aa07037d2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '64d0dfbc6ec64c44a5ac8b652f25e521'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:18.659146 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 973/1984] 10.222.0.55 () {58 vars in 1224 bytes} [Fri Sep 23 15:21:18 2022] DELETE /identity/v3/users/64d0dfbc6ec64c44a5ac8b652f25e521 => generated 0 bytes in 509 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:21:18.663662 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a49ecd5-dfa8-41ea-95dd-83a9c5ef8b73 None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:18.663947 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a49ecd5-dfa8-41ea-95dd-83a9c5ef8b73 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:18.664778 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a49ecd5-dfa8-41ea-95dd-83a9c5ef8b73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:18.675797 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-a3430717-88ca-4ef1-af14-38e862718d1b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b643cc7818c49fb88ac37d962844efb', 'request_id': 'req-a3430717-88ca-4ef1-af14-38e862718d1b'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:18.683035 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-a3430717-88ca-4ef1-af14-38e862718d1b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b643cc7818c49fb88ac37d962844efb', 'request_id': 'req-a3430717-88ca-4ef1-af14-38e862718d1b'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:18.687241 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-5a49ecd5-dfa8-41ea-95dd-83a9c5ef8b73 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:21:18.689171 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-a3430717-88ca-4ef1-af14-38e862718d1b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b643cc7818c49fb88ac37d962844efb', 'request_id': 'req-a3430717-88ca-4ef1-af14-38e862718d1b'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:18.689369 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-5a49ecd5-dfa8-41ea-95dd-83a9c5ef8b73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:18.756521 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 974/1985] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:21:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 15:21:18.767496 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a49ecd5-dfa8-41ea-95dd-83a9c5ef8b73 None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:18.770784 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-5a49ecd5-dfa8-41ea-95dd-83a9c5ef8b73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:18.800567 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-a3430717-88ca-4ef1-af14-38e862718d1b admin admin] The token cache is being invalidate because project 0b643cc7818c49fb88ac37d962844efb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81994) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 15:21:18.800567 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-a3430717-88ca-4ef1-af14-38e862718d1b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:18.801237 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-a3430717-88ca-4ef1-af14-38e862718d1b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:18.803018 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1012/1986] 10.222.0.55 () {58 vars in 1233 bytes} [Fri Sep 23 15:21:18 2022] DELETE /identity/v3/projects/0b643cc7818c49fb88ac37d962844efb => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 15:21:18.808540 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7999d30-2797-4f25-9956-c8c2056cf068 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:18.808830 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7999d30-2797-4f25-9956-c8c2056cf068 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:18.809174 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7999d30-2797-4f25-9956-c8c2056cf068 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:18.833398 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a49ecd5-dfa8-41ea-95dd-83a9c5ef8b73 None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:18.840692 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116f983b-707f-46a7-86b2-bf9a48425ddb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd259178570864ffeb936ee10dfec6b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:18.841356 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116f983b-707f-46a7-86b2-bf9a48425ddb service neutron] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:18.841638 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116f983b-707f-46a7-86b2-bf9a48425ddb service neutron] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:18.841916 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116f983b-707f-46a7-86b2-bf9a48425ddb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:18.843673 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-a7999d30-2797-4f25-9956-c8c2056cf068 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:21:18.843797 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-116f983b-707f-46a7-86b2-bf9a48425ddb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:18.845225 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-a7999d30-2797-4f25-9956-c8c2056cf068 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:18.857174 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116f983b-707f-46a7-86b2-bf9a48425ddb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:18.858832 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116f983b-707f-46a7-86b2-bf9a48425ddb service neutron] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:18.862369 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1013/1987] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:21:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:21:18.863771 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b235234-3e79-4588-a7ff-fb0c0bf10567 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:18.864187 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b235234-3e79-4588-a7ff-fb0c0bf10567 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:18.864376 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b235234-3e79-4588-a7ff-fb0c0bf10567 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:18.870055 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 975/1988] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:21:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 23 15:21:18.896897 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-0b235234-3e79-4588-a7ff-fb0c0bf10567 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:21:18.898167 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-0b235234-3e79-4588-a7ff-fb0c0bf10567 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:18.907320 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1014/1989] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:21:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:21:18.911930 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116f983b-707f-46a7-86b2-bf9a48425ddb service neutron] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:18.923147 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116f983b-707f-46a7-86b2-bf9a48425ddb service neutron] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:18.928047 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e531964-4da2-4885-871d-0b57651c60d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:18.928801 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e531964-4da2-4885-871d-0b57651c60d8 admin admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:18.929079 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e531964-4da2-4885-871d-0b57651c60d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:18.929353 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e531964-4da2-4885-871d-0b57651c60d8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:18.930063 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e531964-4da2-4885-871d-0b57651c60d8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:21:18.930383 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e531964-4da2-4885-871d-0b57651c60d8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:18.932030 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e531964-4da2-4885-871d-0b57651c60d8 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:18.944824 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 976/1990] 10.222.0.55 () {58 vars in 1153 bytes} [Fri Sep 23 15:21:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:21:18.951622 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b235234-3e79-4588-a7ff-fb0c0bf10567 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:18.966186 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b235234-3e79-4588-a7ff-fb0c0bf10567 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:18.971588 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1c9c8b-417b-47d3-8aee-5fbeb3466eb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:18.972310 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1c9c8b-417b-47d3-8aee-5fbeb3466eb2 admin admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:18.972686 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1c9c8b-417b-47d3-8aee-5fbeb3466eb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:18.973031 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1c9c8b-417b-47d3-8aee-5fbeb3466eb2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:18.973754 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1c9c8b-417b-47d3-8aee-5fbeb3466eb2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:18.975879 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1c9c8b-417b-47d3-8aee-5fbeb3466eb2 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:19.163591 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1015/1991] 10.222.0.55 () {60 vars in 1153 bytes} [Fri Sep 23 15:21:18 2022] POST /identity/v3/projects => generated 365 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:21:19.169864 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e531964-4da2-4885-871d-0b57651c60d8 admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:19.180410 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e531964-4da2-4885-871d-0b57651c60d8 admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:19.238698 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de209b4-9cf7-4603-8d53-29d9b77c0889 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:19.239360 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de209b4-9cf7-4603-8d53-29d9b77c0889 admin admin] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:19.239640 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de209b4-9cf7-4603-8d53-29d9b77c0889 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:19.239914 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de209b4-9cf7-4603-8d53-29d9b77c0889 admin admin] PATH_INFO: `/v3/users` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:19.240581 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de209b4-9cf7-4603-8d53-29d9b77c0889 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:19.242552 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de209b4-9cf7-4603-8d53-29d9b77c0889 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:19.362470 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1c9c8b-417b-47d3-8aee-5fbeb3466eb2 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:19.364264 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-4e1c9c8b-417b-47d3-8aee-5fbeb3466eb2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:19.377679 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1c9c8b-417b-47d3-8aee-5fbeb3466eb2 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:19.384398 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5757266f-bc50-4a2a-a14a-2f1e0fa29314 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:19.385810 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5757266f-bc50-4a2a-a14a-2f1e0fa29314 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:19.386445 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5757266f-bc50-4a2a-a14a-2f1e0fa29314 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:19.386728 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5757266f-bc50-4a2a-a14a-2f1e0fa29314 admin admin] PATH_INFO: `/v3/projects/0d437d9063644d7b97ea5cea377dc024` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:19.388188 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5757266f-bc50-4a2a-a14a-2f1e0fa29314 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0d437d9063644d7b97ea5cea377dc024)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:19.390121 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5757266f-bc50-4a2a-a14a-2f1e0fa29314 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:19.402834 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 977/1992] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Sep 23 15:21:19 2022] POST /identity/v3/users => generated 283 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 23 15:21:19.407007 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de209b4-9cf7-4603-8d53-29d9b77c0889 admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:19.417784 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de209b4-9cf7-4603-8d53-29d9b77c0889 admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:19.422397 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7740daa2-8190-4ba5-8208-4ed70bd1ad39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:19.423021 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7740daa2-8190-4ba5-8208-4ed70bd1ad39 admin admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:19.423284 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7740daa2-8190-4ba5-8208-4ed70bd1ad39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:19.423583 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7740daa2-8190-4ba5-8208-4ed70bd1ad39 admin admin] PATH_INFO: `/v3/roles` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:19.424310 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7740daa2-8190-4ba5-8208-4ed70bd1ad39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:19.425901 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7740daa2-8190-4ba5-8208-4ed70bd1ad39 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:19.438296 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 978/1993] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Sep 23 15:21:19 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 23 15:21:19.442060 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7740daa2-8190-4ba5-8208-4ed70bd1ad39 admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:19.451656 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7740daa2-8190-4ba5-8208-4ed70bd1ad39 admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:19.456283 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181017f8-78eb-435e-9600-481cc9a74c36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:19.456862 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181017f8-78eb-435e-9600-481cc9a74c36 admin admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:19.457140 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181017f8-78eb-435e-9600-481cc9a74c36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:19.457412 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181017f8-78eb-435e-9600-481cc9a74c36 admin admin] PATH_INFO: `/v3/roles` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:19.458044 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181017f8-78eb-435e-9600-481cc9a74c36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:19.459598 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181017f8-78eb-435e-9600-481cc9a74c36 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:19.470086 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 979/1994] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Sep 23 15:21:19 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 23 15:21:19.474144 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181017f8-78eb-435e-9600-481cc9a74c36 admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:19.484643 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181017f8-78eb-435e-9600-481cc9a74c36 admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:19.489242 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f421182-6195-482d-96c1-64501d6e0813 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:19.489866 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f421182-6195-482d-96c1-64501d6e0813 admin admin] REQUEST_METHOD: `PUT` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:19.490149 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f421182-6195-482d-96c1-64501d6e0813 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:19.490414 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f421182-6195-482d-96c1-64501d6e0813 admin admin] PATH_INFO: `/v3/projects/9e93e2850373495cb1abd9a04f30d142/users/1c5f5561caaa4223ad43f95bba10e739/roles/af257be2dee54d6b942f58650196582c` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:19.505060 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f421182-6195-482d-96c1-64501d6e0813 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e93e2850373495cb1abd9a04f30d142, user_id=1c5f5561caaa4223ad43f95bba10e739, role_id=af257be2dee54d6b942f58650196582c)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:19.507860 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f421182-6195-482d-96c1-64501d6e0813 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:19.618661 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 980/1995] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Sep 23 15:21:19 2022] PUT /identity/v3/projects/9e93e2850373495cb1abd9a04f30d142/users/1c5f5561caaa4223ad43f95bba10e739/roles/af257be2dee54d6b942f58650196582c => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 15:21:19.629118 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d7e753-5298-4272-af06-69399d9b34f1 None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:19.629444 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d7e753-5298-4272-af06-69399d9b34f1 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:19.629793 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d7e753-5298-4272-af06-69399d9b34f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:19.638099 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-5757266f-bc50-4a2a-a14a-2f1e0fa29314 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0d437d9063644d7b97ea5cea377dc024', 'request_id': 'req-5757266f-bc50-4a2a-a14a-2f1e0fa29314'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:19.644855 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-5757266f-bc50-4a2a-a14a-2f1e0fa29314 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0d437d9063644d7b97ea5cea377dc024', 'request_id': 'req-5757266f-bc50-4a2a-a14a-2f1e0fa29314'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:19.652029 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-5757266f-bc50-4a2a-a14a-2f1e0fa29314 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0d437d9063644d7b97ea5cea377dc024', 'request_id': 'req-5757266f-bc50-4a2a-a14a-2f1e0fa29314'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:19.662419 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-71d7e753-5298-4272-af06-69399d9b34f1 None None] MFA Rules not processed for user `1c5f5561caaa4223ad43f95bba10e739`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:21:19.663783 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-71d7e753-5298-4272-af06-69399d9b34f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:19.732317 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 981/1996] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:21:19 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:21:19.744401 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d7e753-5298-4272-af06-69399d9b34f1 None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:19.745768 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-71d7e753-5298-4272-af06-69399d9b34f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:19.754782 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-5757266f-bc50-4a2a-a14a-2f1e0fa29314 admin admin] The token cache is being invalidate because project 0d437d9063644d7b97ea5cea377dc024 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81994) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 15:21:19.755077 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-5757266f-bc50-4a2a-a14a-2f1e0fa29314 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:19.755697 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-5757266f-bc50-4a2a-a14a-2f1e0fa29314 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:19.757198 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1016/1997] 10.222.0.55 () {58 vars in 1233 bytes} [Fri Sep 23 15:21:19 2022] DELETE /identity/v3/projects/0d437d9063644d7b97ea5cea377dc024 => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 15:21:19.761351 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5757266f-bc50-4a2a-a14a-2f1e0fa29314 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:19.762941 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-5757266f-bc50-4a2a-a14a-2f1e0fa29314 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:19.806200 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d7e753-5298-4272-af06-69399d9b34f1 None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:19.809989 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0656a7c1-14ed-4381-957f-4ac40a4c46fc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feca6278be904ef8b0dc580a67e477f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:19.810609 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0656a7c1-14ed-4381-957f-4ac40a4c46fc service cinder] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:19.810888 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0656a7c1-14ed-4381-957f-4ac40a4c46fc service cinder] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:19.811167 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0656a7c1-14ed-4381-957f-4ac40a4c46fc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:19.813051 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-0656a7c1-14ed-4381-957f-4ac40a4c46fc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:19.824925 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0656a7c1-14ed-4381-957f-4ac40a4c46fc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:19.829913 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0656a7c1-14ed-4381-957f-4ac40a4c46fc service cinder] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:19.830080 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5757266f-bc50-4a2a-a14a-2f1e0fa29314 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:19.833905 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02ef6d3-5279-450c-ba9e-e75b9e244661 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:19.834539 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02ef6d3-5279-450c-ba9e-e75b9e244661 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:19.834822 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02ef6d3-5279-450c-ba9e-e75b9e244661 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:19.835099 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02ef6d3-5279-450c-ba9e-e75b9e244661 admin admin] PATH_INFO: `/v3/users/1542fa130b944a67b4ea2fefb7aa48fe` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:19.835550 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 982/1998] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:21:19 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 23 15:21:19.837038 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02ef6d3-5279-450c-ba9e-e75b9e244661 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1542fa130b944a67b4ea2fefb7aa48fe)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:19.839000 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02ef6d3-5279-450c-ba9e-e75b9e244661 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:20.099353 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-b02ef6d3-5279-450c-ba9e-e75b9e244661 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1542fa130b944a67b4ea2fefb7aa48fe'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:20.122323 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-b02ef6d3-5279-450c-ba9e-e75b9e244661 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1542fa130b944a67b4ea2fefb7aa48fe'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:20.122323 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-b02ef6d3-5279-450c-ba9e-e75b9e244661 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1542fa130b944a67b4ea2fefb7aa48fe'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:20.275868 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-b02ef6d3-5279-450c-ba9e-e75b9e244661 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1542fa130b944a67b4ea2fefb7aa48fe'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:20.276192 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-b02ef6d3-5279-450c-ba9e-e75b9e244661 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1542fa130b944a67b4ea2fefb7aa48fe'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:20.282774 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1017/1999] 10.222.0.55 () {58 vars in 1224 bytes} [Fri Sep 23 15:21:19 2022] DELETE /identity/v3/users/1542fa130b944a67b4ea2fefb7aa48fe => generated 0 bytes in 522 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 15:21:20.682086 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:21:20.911252 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0656a7c1-14ed-4381-957f-4ac40a4c46fc service cinder] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:20.912842 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-0656a7c1-14ed-4381-957f-4ac40a4c46fc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:20.924072 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0656a7c1-14ed-4381-957f-4ac40a4c46fc service cinder] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:20.950140 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02ef6d3-5279-450c-ba9e-e75b9e244661 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:20.951889 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-b02ef6d3-5279-450c-ba9e-e75b9e244661 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:20.983525 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8933730-e6d5-437d-a9ee-80c6295d7682 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feca6278be904ef8b0dc580a67e477f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:20.984197 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8933730-e6d5-437d-a9ee-80c6295d7682 service cinder] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:20.984600 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8933730-e6d5-437d-a9ee-80c6295d7682 service cinder] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:20.984885 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8933730-e6d5-437d-a9ee-80c6295d7682 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:20.987240 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-a8933730-e6d5-437d-a9ee-80c6295d7682 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:21.026117 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02ef6d3-5279-450c-ba9e-e75b9e244661 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:21.031229 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5fe8d9-49bb-4474-9397-a5a2b8f60c4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:21.031683 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5fe8d9-49bb-4474-9397-a5a2b8f60c4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:21.032049 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5fe8d9-49bb-4474-9397-a5a2b8f60c4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:21.032329 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5fe8d9-49bb-4474-9397-a5a2b8f60c4e admin admin] PATH_INFO: `/v3/projects/8eda34d3f24a429ca2627b289a2935d2` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:21.033507 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5fe8d9-49bb-4474-9397-a5a2b8f60c4e admin admin] RBAC: Authorizing `identity:delete_project(project_id=8eda34d3f24a429ca2627b289a2935d2)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:21.035422 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5fe8d9-49bb-4474-9397-a5a2b8f60c4e admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:21.060130 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8933730-e6d5-437d-a9ee-80c6295d7682 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:21.060910 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8933730-e6d5-437d-a9ee-80c6295d7682 service cinder] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:21.083531 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 983/2000] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:21:20 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:21:21.208090 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-2f5fe8d9-49bb-4474-9397-a5a2b8f60c4e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8eda34d3f24a429ca2627b289a2935d2', 'request_id': 'req-2f5fe8d9-49bb-4474-9397-a5a2b8f60c4e'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:21.215689 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-2f5fe8d9-49bb-4474-9397-a5a2b8f60c4e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8eda34d3f24a429ca2627b289a2935d2', 'request_id': 'req-2f5fe8d9-49bb-4474-9397-a5a2b8f60c4e'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:21.222945 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-2f5fe8d9-49bb-4474-9397-a5a2b8f60c4e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8eda34d3f24a429ca2627b289a2935d2', 'request_id': 'req-2f5fe8d9-49bb-4474-9397-a5a2b8f60c4e'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:21.381611 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-2f5fe8d9-49bb-4474-9397-a5a2b8f60c4e admin admin] The token cache is being invalidate because project 8eda34d3f24a429ca2627b289a2935d2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81994) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 15:21:21.381896 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-2f5fe8d9-49bb-4474-9397-a5a2b8f60c4e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:21.384584 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-2f5fe8d9-49bb-4474-9397-a5a2b8f60c4e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:21.384804 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1018/2001] 10.222.0.55 () {58 vars in 1233 bytes} [Fri Sep 23 15:21:20 2022] DELETE /identity/v3/projects/8eda34d3f24a429ca2627b289a2935d2 => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 15:21:21.390426 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66ade68-dc59-4f2a-be1a-54083c9260f8 None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:21.390713 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66ade68-dc59-4f2a-be1a-54083c9260f8 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:21.390986 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66ade68-dc59-4f2a-be1a-54083c9260f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:21.411960 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-b66ade68-dc59-4f2a-be1a-54083c9260f8 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:21:21.413283 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-b66ade68-dc59-4f2a-be1a-54083c9260f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:21.476058 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 984/2002] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:21:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:21:21.484591 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6156f36b-6d93-4756-bda2-9281186dbfc7 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:21.484881 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6156f36b-6d93-4756-bda2-9281186dbfc7 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:21.485280 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6156f36b-6d93-4756-bda2-9281186dbfc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:21.510268 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-6156f36b-6d93-4756-bda2-9281186dbfc7 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:21:21.511743 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-6156f36b-6d93-4756-bda2-9281186dbfc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:21.520426 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1019/2003] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:21:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:21:21.528409 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66ade68-dc59-4f2a-be1a-54083c9260f8 None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:21.538256 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66ade68-dc59-4f2a-be1a-54083c9260f8 None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:21.543244 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23ee5e3-d595-4bee-be11-fafb7d37175b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:21.543857 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23ee5e3-d595-4bee-be11-fafb7d37175b admin admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:21.544178 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23ee5e3-d595-4bee-be11-fafb7d37175b admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:21.544474 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23ee5e3-d595-4bee-be11-fafb7d37175b admin admin] PATH_INFO: `/v3/domains` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:21.545119 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23ee5e3-d595-4bee-be11-fafb7d37175b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:21:21.545420 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23ee5e3-d595-4bee-be11-fafb7d37175b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:21.547013 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23ee5e3-d595-4bee-be11-fafb7d37175b admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:21.559304 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 985/2004] 10.222.0.55 () {58 vars in 1153 bytes} [Fri Sep 23 15:21:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 15:21:21.563810 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6156f36b-6d93-4756-bda2-9281186dbfc7 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:21.574131 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6156f36b-6d93-4756-bda2-9281186dbfc7 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:21.579351 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314bd9d6-be11-4756-b18f-350aa88726b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:21.580176 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314bd9d6-be11-4756-b18f-350aa88726b0 admin admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:21.580461 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314bd9d6-be11-4756-b18f-350aa88726b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:21.580794 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314bd9d6-be11-4756-b18f-350aa88726b0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:21.581436 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314bd9d6-be11-4756-b18f-350aa88726b0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:21.583510 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314bd9d6-be11-4756-b18f-350aa88726b0 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:21.593237 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:21:21.722057 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1020/2005] 10.222.0.55 () {60 vars in 1153 bytes} [Fri Sep 23 15:21:21 2022] POST /identity/v3/projects => generated 387 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:21:21.726285 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23ee5e3-d595-4bee-be11-fafb7d37175b admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:21.736763 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23ee5e3-d595-4bee-be11-fafb7d37175b admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:21.798631 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a418a80-bc8b-43ba-9bee-96846eb779cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:21.799266 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a418a80-bc8b-43ba-9bee-96846eb779cd admin admin] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:21.799550 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a418a80-bc8b-43ba-9bee-96846eb779cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:21.799961 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a418a80-bc8b-43ba-9bee-96846eb779cd admin admin] PATH_INFO: `/v3/users` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:21.800684 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a418a80-bc8b-43ba-9bee-96846eb779cd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:21.802695 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a418a80-bc8b-43ba-9bee-96846eb779cd admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:21.909871 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 986/2006] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Sep 23 15:21:21 2022] POST /identity/v3/users => generated 294 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:21:21.913943 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314bd9d6-be11-4756-b18f-350aa88726b0 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:21.925319 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314bd9d6-be11-4756-b18f-350aa88726b0 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:21.930602 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c709f43-1617-436e-b5da-c046b8ae8b08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:21.931439 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c709f43-1617-436e-b5da-c046b8ae8b08 admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:21.931720 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c709f43-1617-436e-b5da-c046b8ae8b08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:21.932012 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c709f43-1617-436e-b5da-c046b8ae8b08 admin admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:21.932657 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c709f43-1617-436e-b5da-c046b8ae8b08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:21.934278 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c709f43-1617-436e-b5da-c046b8ae8b08 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:21.944223 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1021/2007] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Sep 23 15:21:21 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:21:21.950283 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a418a80-bc8b-43ba-9bee-96846eb779cd admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:21.960452 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a418a80-bc8b-43ba-9bee-96846eb779cd admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:21.967217 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bc44ca-2c62-497c-ae3b-28c505d9a539 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:21.967835 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bc44ca-2c62-497c-ae3b-28c505d9a539 admin admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:21.968143 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bc44ca-2c62-497c-ae3b-28c505d9a539 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:21.968417 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bc44ca-2c62-497c-ae3b-28c505d9a539 admin admin] PATH_INFO: `/v3/roles` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:21.969063 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bc44ca-2c62-497c-ae3b-28c505d9a539 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:21.970711 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bc44ca-2c62-497c-ae3b-28c505d9a539 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:21.987751 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 987/2008] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Sep 23 15:21:21 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:21:21.991167 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c709f43-1617-436e-b5da-c046b8ae8b08 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:22.004685 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c709f43-1617-436e-b5da-c046b8ae8b08 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:22.011275 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868ea952-db6e-48e7-b4ba-01ed8e6de8fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:22.012119 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868ea952-db6e-48e7-b4ba-01ed8e6de8fd admin admin] REQUEST_METHOD: `PUT` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:22.012638 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868ea952-db6e-48e7-b4ba-01ed8e6de8fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:22.012812 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868ea952-db6e-48e7-b4ba-01ed8e6de8fd admin admin] PATH_INFO: `/v3/projects/e6adbe60c7ea4978987101a4f9c6b14d/users/7da63af404704b0a959a030aa387cf32/roles/af257be2dee54d6b942f58650196582c` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:22.030288 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868ea952-db6e-48e7-b4ba-01ed8e6de8fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=e6adbe60c7ea4978987101a4f9c6b14d, user_id=7da63af404704b0a959a030aa387cf32, role_id=af257be2dee54d6b942f58650196582c)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:22.033032 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868ea952-db6e-48e7-b4ba-01ed8e6de8fd admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:22.138926 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1022/2009] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Sep 23 15:21:21 2022] PUT /identity/v3/projects/e6adbe60c7ea4978987101a4f9c6b14d/users/7da63af404704b0a959a030aa387cf32/roles/af257be2dee54d6b942f58650196582c => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 15:21:22.151031 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6efb37-b4ef-4b5b-bd59-690a4edbc42f None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:22.151348 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6efb37-b4ef-4b5b-bd59-690a4edbc42f None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:22.151706 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6efb37-b4ef-4b5b-bd59-690a4edbc42f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:22.186667 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-ae6efb37-b4ef-4b5b-bd59-690a4edbc42f None None] MFA Rules not processed for user `7da63af404704b0a959a030aa387cf32`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:21:22.187816 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-ae6efb37-b4ef-4b5b-bd59-690a4edbc42f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:22.266182 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 988/2010] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:21:22 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:21:22.270790 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868ea952-db6e-48e7-b4ba-01ed8e6de8fd admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:22.351269 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868ea952-db6e-48e7-b4ba-01ed8e6de8fd admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:22.357060 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19d19f2-8ab8-45d9-b984-64e6446184fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:22.357955 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19d19f2-8ab8-45d9-b984-64e6446184fe admin admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:22.358465 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19d19f2-8ab8-45d9-b984-64e6446184fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:22.358991 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19d19f2-8ab8-45d9-b984-64e6446184fe admin admin] PATH_INFO: `/v3/projects` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:22.360137 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19d19f2-8ab8-45d9-b984-64e6446184fe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:22.362895 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19d19f2-8ab8-45d9-b984-64e6446184fe admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:22.540208 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1023/2011] 10.222.0.55 () {60 vars in 1153 bytes} [Fri Sep 23 15:21:22 2022] POST /identity/v3/projects => generated 389 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:21:22.544723 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6efb37-b4ef-4b5b-bd59-690a4edbc42f None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:22.556047 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6efb37-b4ef-4b5b-bd59-690a4edbc42f None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:22.615004 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2658a0-7433-4363-83f6-892af84a2ff5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:22.615527 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2658a0-7433-4363-83f6-892af84a2ff5 admin admin] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:22.615804 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2658a0-7433-4363-83f6-892af84a2ff5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:22.616107 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2658a0-7433-4363-83f6-892af84a2ff5 admin admin] PATH_INFO: `/v3/users` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:22.616725 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2658a0-7433-4363-83f6-892af84a2ff5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:22.618666 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2658a0-7433-4363-83f6-892af84a2ff5 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:22.735532 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 989/2012] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Sep 23 15:21:22 2022] POST /identity/v3/users => generated 301 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:21:22.743435 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19d19f2-8ab8-45d9-b984-64e6446184fe admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:22.753629 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19d19f2-8ab8-45d9-b984-64e6446184fe admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:22.758278 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4d0b9d-0029-4c5b-acf6-8805567f1aee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:22.758949 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4d0b9d-0029-4c5b-acf6-8805567f1aee admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:22.759235 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4d0b9d-0029-4c5b-acf6-8805567f1aee admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:22.759512 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4d0b9d-0029-4c5b-acf6-8805567f1aee admin admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:22.760191 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4d0b9d-0029-4c5b-acf6-8805567f1aee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:22.761800 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4d0b9d-0029-4c5b-acf6-8805567f1aee admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:22.772596 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1024/2013] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Sep 23 15:21:22 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:21:22.778406 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2658a0-7433-4363-83f6-892af84a2ff5 admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:22.788749 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2658a0-7433-4363-83f6-892af84a2ff5 admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:22.794404 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d617cf4e-a2df-442f-9142-4bc37f7d55cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:22.795097 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d617cf4e-a2df-442f-9142-4bc37f7d55cb admin admin] REQUEST_METHOD: `PUT` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:22.795375 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d617cf4e-a2df-442f-9142-4bc37f7d55cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:22.795648 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d617cf4e-a2df-442f-9142-4bc37f7d55cb admin admin] PATH_INFO: `/v3/projects/1a452cc54b8e4e24a26c630e01941303/users/ca6e02671a42422f85ac729656b3844f/roles/8e2919e79df040e8926f21607818b58c` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:22.811008 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d617cf4e-a2df-442f-9142-4bc37f7d55cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a452cc54b8e4e24a26c630e01941303, user_id=ca6e02671a42422f85ac729656b3844f, role_id=8e2919e79df040e8926f21607818b58c)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:22.812059 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:21:22.815169 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d617cf4e-a2df-442f-9142-4bc37f7d55cb admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:22.909212 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 990/2014] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Sep 23 15:21:22 2022] PUT /identity/v3/projects/1a452cc54b8e4e24a26c630e01941303/users/ca6e02671a42422f85ac729656b3844f/roles/8e2919e79df040e8926f21607818b58c => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 15:21:22.913504 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4d0b9d-0029-4c5b-acf6-8805567f1aee admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:22.924371 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4d0b9d-0029-4c5b-acf6-8805567f1aee admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:22.989838 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292a6ffa-299f-43b5-a7c5-58ba59a208f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:22.989838 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292a6ffa-299f-43b5-a7c5-58ba59a208f7 admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:22.989838 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292a6ffa-299f-43b5-a7c5-58ba59a208f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:22.989838 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292a6ffa-299f-43b5-a7c5-58ba59a208f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:22.990460 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292a6ffa-299f-43b5-a7c5-58ba59a208f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:22.992142 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292a6ffa-299f-43b5-a7c5-58ba59a208f7 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:23.002744 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1025/2015] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Sep 23 15:21:22 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:21:23.006770 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d617cf4e-a2df-442f-9142-4bc37f7d55cb admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:23.018266 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d617cf4e-a2df-442f-9142-4bc37f7d55cb admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:23.023219 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da543ac3-d875-46c4-8b3d-7df5eade2450 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:23.024889 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da543ac3-d875-46c4-8b3d-7df5eade2450 admin admin] REQUEST_METHOD: `PUT` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:23.025404 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da543ac3-d875-46c4-8b3d-7df5eade2450 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:23.025688 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da543ac3-d875-46c4-8b3d-7df5eade2450 admin admin] PATH_INFO: `/v3/projects/1a452cc54b8e4e24a26c630e01941303/users/ca6e02671a42422f85ac729656b3844f/roles/8e2919e79df040e8926f21607818b58c` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:23.027684 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da543ac3-d875-46c4-8b3d-7df5eade2450 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a452cc54b8e4e24a26c630e01941303, user_id=ca6e02671a42422f85ac729656b3844f, role_id=8e2919e79df040e8926f21607818b58c)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:23.030439 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da543ac3-d875-46c4-8b3d-7df5eade2450 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:23.047820 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 991/2016] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Sep 23 15:21:23 2022] PUT /identity/v3/projects/1a452cc54b8e4e24a26c630e01941303/users/ca6e02671a42422f85ac729656b3844f/roles/8e2919e79df040e8926f21607818b58c => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:21:23.057337 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8baac9e-5d15-4332-9379-cef676d2c5cd None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:23.057629 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8baac9e-5d15-4332-9379-cef676d2c5cd None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:23.057906 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8baac9e-5d15-4332-9379-cef676d2c5cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:23.089300 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-e8baac9e-5d15-4332-9379-cef676d2c5cd None None] MFA Rules not processed for user `ca6e02671a42422f85ac729656b3844f`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:21:23.090595 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-e8baac9e-5d15-4332-9379-cef676d2c5cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:23.162667 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1026/2017] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:21:23 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:21:23.173934 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da543ac3-d875-46c4-8b3d-7df5eade2450 admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:23.176236 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-da543ac3-d875-46c4-8b3d-7df5eade2450 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:23.234497 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da543ac3-d875-46c4-8b3d-7df5eade2450 admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:23.238267 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6211da00-4967-4dea-88e3-e93f0b65a439 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feca6278be904ef8b0dc580a67e477f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:23.238880 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6211da00-4967-4dea-88e3-e93f0b65a439 service cinder] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:23.239164 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6211da00-4967-4dea-88e3-e93f0b65a439 service cinder] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:23.239440 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6211da00-4967-4dea-88e3-e93f0b65a439 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:23.248533 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6211da00-4967-4dea-88e3-e93f0b65a439 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:23.249956 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6211da00-4967-4dea-88e3-e93f0b65a439 service cinder] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:23.259559 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 992/2018] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:21:23 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:21:23.311557 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:21:23.409845 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8baac9e-5d15-4332-9379-cef676d2c5cd None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:23.479890 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8baac9e-5d15-4332-9379-cef676d2c5cd None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:23.484515 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729364dd-a6fd-4e13-9cf2-a43ac0c291b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:23.485147 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729364dd-a6fd-4e13-9cf2-a43ac0c291b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:23.485424 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729364dd-a6fd-4e13-9cf2-a43ac0c291b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:23.485701 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729364dd-a6fd-4e13-9cf2-a43ac0c291b1 admin admin] PATH_INFO: `/v3/users/7da63af404704b0a959a030aa387cf32` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:23.486821 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729364dd-a6fd-4e13-9cf2-a43ac0c291b1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7da63af404704b0a959a030aa387cf32)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:23.489410 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729364dd-a6fd-4e13-9cf2-a43ac0c291b1 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:23.740466 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-729364dd-a6fd-4e13-9cf2-a43ac0c291b1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7da63af404704b0a959a030aa387cf32'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:23.754229 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-729364dd-a6fd-4e13-9cf2-a43ac0c291b1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7da63af404704b0a959a030aa387cf32'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:23.763907 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-729364dd-a6fd-4e13-9cf2-a43ac0c291b1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7da63af404704b0a959a030aa387cf32'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:23.880338 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-729364dd-a6fd-4e13-9cf2-a43ac0c291b1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7da63af404704b0a959a030aa387cf32'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:23.881123 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-729364dd-a6fd-4e13-9cf2-a43ac0c291b1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7da63af404704b0a959a030aa387cf32'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:23.888606 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1027/2019] 10.222.0.55 () {58 vars in 1224 bytes} [Fri Sep 23 15:21:23 2022] DELETE /identity/v3/users/7da63af404704b0a959a030aa387cf32 => generated 0 bytes in 479 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:21:23.893078 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ca2402-8883-49cf-b7a1-c61673408d76 None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:23.893358 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ca2402-8883-49cf-b7a1-c61673408d76 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:23.893655 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ca2402-8883-49cf-b7a1-c61673408d76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:23.913479 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-43ca2402-8883-49cf-b7a1-c61673408d76 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:21:23.915287 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-43ca2402-8883-49cf-b7a1-c61673408d76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:23.978477 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 993/2020] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:21:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:21:23.988577 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729364dd-a6fd-4e13-9cf2-a43ac0c291b1 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:23.990020 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-729364dd-a6fd-4e13-9cf2-a43ac0c291b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:24.049959 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729364dd-a6fd-4e13-9cf2-a43ac0c291b1 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:24.053702 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d869ad-f202-47eb-84e2-edeae619db88 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd259178570864ffeb936ee10dfec6b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:24.054315 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d869ad-f202-47eb-84e2-edeae619db88 service neutron] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:24.054588 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d869ad-f202-47eb-84e2-edeae619db88 service neutron] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:24.055063 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d869ad-f202-47eb-84e2-edeae619db88 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:24.068098 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d869ad-f202-47eb-84e2-edeae619db88 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:24.069137 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d869ad-f202-47eb-84e2-edeae619db88 service neutron] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:24.088497 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1028/2021] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:21:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:21:24.286440 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:21:24.527305 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:21:24.534147 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ca2402-8883-49cf-b7a1-c61673408d76 None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:24.535596 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-43ca2402-8883-49cf-b7a1-c61673408d76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:24.549124 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ca2402-8883-49cf-b7a1-c61673408d76 None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:24.554149 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b38569-3f85-4181-bf2f-cee2b65ca8fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:24.554794 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b38569-3f85-4181-bf2f-cee2b65ca8fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:24.555082 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b38569-3f85-4181-bf2f-cee2b65ca8fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:24.555486 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b38569-3f85-4181-bf2f-cee2b65ca8fe admin admin] PATH_INFO: `/v3/projects/e6adbe60c7ea4978987101a4f9c6b14d` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:24.556792 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b38569-3f85-4181-bf2f-cee2b65ca8fe admin admin] RBAC: Authorizing `identity:delete_project(project_id=e6adbe60c7ea4978987101a4f9c6b14d)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:24.558680 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b38569-3f85-4181-bf2f-cee2b65ca8fe admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:24.833069 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-67b38569-3f85-4181-bf2f-cee2b65ca8fe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e6adbe60c7ea4978987101a4f9c6b14d', 'request_id': 'req-67b38569-3f85-4181-bf2f-cee2b65ca8fe'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:24.838481 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-67b38569-3f85-4181-bf2f-cee2b65ca8fe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e6adbe60c7ea4978987101a4f9c6b14d', 'request_id': 'req-67b38569-3f85-4181-bf2f-cee2b65ca8fe'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:24.937435 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-67b38569-3f85-4181-bf2f-cee2b65ca8fe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e6adbe60c7ea4978987101a4f9c6b14d', 'request_id': 'req-67b38569-3f85-4181-bf2f-cee2b65ca8fe'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:24.950089 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-67b38569-3f85-4181-bf2f-cee2b65ca8fe admin admin] The token cache is being invalidate because project e6adbe60c7ea4978987101a4f9c6b14d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81993) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 15:21:24.950379 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-67b38569-3f85-4181-bf2f-cee2b65ca8fe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:24.951018 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-67b38569-3f85-4181-bf2f-cee2b65ca8fe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:24.952634 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 994/2022] 10.222.0.55 () {58 vars in 1233 bytes} [Fri Sep 23 15:21:24 2022] DELETE /identity/v3/projects/e6adbe60c7ea4978987101a4f9c6b14d => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:21:24.956713 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d869ad-f202-47eb-84e2-edeae619db88 service neutron] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:24.967014 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d869ad-f202-47eb-84e2-edeae619db88 service neutron] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:25.024563 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60562a6a-d254-4840-a69d-0f5a6f039b49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:25.025248 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60562a6a-d254-4840-a69d-0f5a6f039b49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:25.025524 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60562a6a-d254-4840-a69d-0f5a6f039b49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:25.025797 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60562a6a-d254-4840-a69d-0f5a6f039b49 admin admin] PATH_INFO: `/v3/users/ca6e02671a42422f85ac729656b3844f` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:25.026929 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60562a6a-d254-4840-a69d-0f5a6f039b49 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ca6e02671a42422f85ac729656b3844f)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:25.029045 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60562a6a-d254-4840-a69d-0f5a6f039b49 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:25.270157 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-60562a6a-d254-4840-a69d-0f5a6f039b49 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ca6e02671a42422f85ac729656b3844f'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:25.285980 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-60562a6a-d254-4840-a69d-0f5a6f039b49 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ca6e02671a42422f85ac729656b3844f'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:25.294200 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-60562a6a-d254-4840-a69d-0f5a6f039b49 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ca6e02671a42422f85ac729656b3844f'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:25.413456 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-60562a6a-d254-4840-a69d-0f5a6f039b49 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ca6e02671a42422f85ac729656b3844f'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:25.414201 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-60562a6a-d254-4840-a69d-0f5a6f039b49 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ca6e02671a42422f85ac729656b3844f'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:25.422422 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1029/2023] 10.222.0.55 () {58 vars in 1224 bytes} [Fri Sep 23 15:21:24 2022] DELETE /identity/v3/users/ca6e02671a42422f85ac729656b3844f => generated 0 bytes in 466 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:21:26.018449 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b38569-3f85-4181-bf2f-cee2b65ca8fe admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:26.020325 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-67b38569-3f85-4181-bf2f-cee2b65ca8fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:26.035422 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b38569-3f85-4181-bf2f-cee2b65ca8fe admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:26.098786 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f338e9d6-5be0-4927-8a97-04267b36bc44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:26.100012 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f338e9d6-5be0-4927-8a97-04267b36bc44 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:26.100012 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f338e9d6-5be0-4927-8a97-04267b36bc44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:26.100012 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f338e9d6-5be0-4927-8a97-04267b36bc44 admin admin] PATH_INFO: `/v3/projects/1a452cc54b8e4e24a26c630e01941303` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:26.101497 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f338e9d6-5be0-4927-8a97-04267b36bc44 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1a452cc54b8e4e24a26c630e01941303)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:26.104362 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f338e9d6-5be0-4927-8a97-04267b36bc44 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:26.267018 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-f338e9d6-5be0-4927-8a97-04267b36bc44 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1a452cc54b8e4e24a26c630e01941303', 'request_id': 'req-f338e9d6-5be0-4927-8a97-04267b36bc44'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:26.272420 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-f338e9d6-5be0-4927-8a97-04267b36bc44 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1a452cc54b8e4e24a26c630e01941303', 'request_id': 'req-f338e9d6-5be0-4927-8a97-04267b36bc44'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:26.366115 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-f338e9d6-5be0-4927-8a97-04267b36bc44 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1a452cc54b8e4e24a26c630e01941303', 'request_id': 'req-f338e9d6-5be0-4927-8a97-04267b36bc44'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:26.390064 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-f338e9d6-5be0-4927-8a97-04267b36bc44 admin admin] The token cache is being invalidate because project 1a452cc54b8e4e24a26c630e01941303 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81993) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 15:21:26.391039 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-f338e9d6-5be0-4927-8a97-04267b36bc44 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:26.391961 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-f338e9d6-5be0-4927-8a97-04267b36bc44 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:26.394210 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 995/2024] 10.222.0.55 () {58 vars in 1233 bytes} [Fri Sep 23 15:21:26 2022] DELETE /identity/v3/projects/1a452cc54b8e4e24a26c630e01941303 => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:21:26.411449 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41efc40d-aa6b-4e29-b26c-31a9879bd680 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:26.412422 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41efc40d-aa6b-4e29-b26c-31a9879bd680 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:26.412881 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41efc40d-aa6b-4e29-b26c-31a9879bd680 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:26.443842 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-41efc40d-aa6b-4e29-b26c-31a9879bd680 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:21:26.443842 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-41efc40d-aa6b-4e29-b26c-31a9879bd680 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:26.471934 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:21:26.512903 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1030/2025] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:21:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:21:26.519631 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbff682-4fa0-4431-b1d5-dced410ac7b4 None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:26.519814 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbff682-4fa0-4431-b1d5-dced410ac7b4 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:26.520125 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbff682-4fa0-4431-b1d5-dced410ac7b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:26.540008 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-2cbff682-4fa0-4431-b1d5-dced410ac7b4 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:21:26.541766 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-2cbff682-4fa0-4431-b1d5-dced410ac7b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:26.549738 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 996/2026] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:21:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:21:26.555695 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41efc40d-aa6b-4e29-b26c-31a9879bd680 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:26.566113 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41efc40d-aa6b-4e29-b26c-31a9879bd680 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:26.570940 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a16557-2db3-4249-9587-792f16116f75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:26.571542 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a16557-2db3-4249-9587-792f16116f75 admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:26.571885 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a16557-2db3-4249-9587-792f16116f75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:26.572439 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a16557-2db3-4249-9587-792f16116f75 admin admin] PATH_INFO: `/v3/domains` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:26.573105 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a16557-2db3-4249-9587-792f16116f75 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:21:26.573415 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a16557-2db3-4249-9587-792f16116f75 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:26.575170 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a16557-2db3-4249-9587-792f16116f75 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:26.587218 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1031/2027] 10.222.0.55 () {58 vars in 1153 bytes} [Fri Sep 23 15:21:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 15:21:26.591683 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbff682-4fa0-4431-b1d5-dced410ac7b4 None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:26.601167 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbff682-4fa0-4431-b1d5-dced410ac7b4 None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:26.608429 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6da4864-400d-4279-856d-e1204cc97488 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:26.609054 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6da4864-400d-4279-856d-e1204cc97488 admin admin] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:26.609338 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6da4864-400d-4279-856d-e1204cc97488 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:26.609614 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6da4864-400d-4279-856d-e1204cc97488 admin admin] PATH_INFO: `/v3/projects` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:26.610265 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6da4864-400d-4279-856d-e1204cc97488 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:26.612276 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6da4864-400d-4279-856d-e1204cc97488 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:26.724382 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 997/2028] 10.222.0.55 () {60 vars in 1153 bytes} [Fri Sep 23 15:21:26 2022] POST /identity/v3/projects => generated 407 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:21:26.730714 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a16557-2db3-4249-9587-792f16116f75 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:26.741866 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a16557-2db3-4249-9587-792f16116f75 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:26.808600 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92796c36-5c9f-458f-9d42-b1c6e04a328d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:26.809210 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92796c36-5c9f-458f-9d42-b1c6e04a328d admin admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:26.809494 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92796c36-5c9f-458f-9d42-b1c6e04a328d admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:26.809771 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92796c36-5c9f-458f-9d42-b1c6e04a328d admin admin] PATH_INFO: `/v3/users` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:26.810444 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92796c36-5c9f-458f-9d42-b1c6e04a328d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:26.812457 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92796c36-5c9f-458f-9d42-b1c6e04a328d admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:26.918075 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6da4864-400d-4279-856d-e1204cc97488 admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:26.919831 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-f6da4864-400d-4279-856d-e1204cc97488 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:26.933042 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6da4864-400d-4279-856d-e1204cc97488 admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:26.937789 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ec72a2-7fef-4090-9a1b-52de8138b2f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:26.938471 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ec72a2-7fef-4090-9a1b-52de8138b2f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:26.938750 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ec72a2-7fef-4090-9a1b-52de8138b2f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:26.939045 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ec72a2-7fef-4090-9a1b-52de8138b2f5 admin admin] PATH_INFO: `/v3/users/5c857ee2049543479944da34eda79b5c` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:26.940299 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ec72a2-7fef-4090-9a1b-52de8138b2f5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5c857ee2049543479944da34eda79b5c)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:26.942348 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ec72a2-7fef-4090-9a1b-52de8138b2f5 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:26.987322 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1032/2029] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Sep 23 15:21:26 2022] POST /identity/v3/users => generated 304 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:21:26.992411 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92796c36-5c9f-458f-9d42-b1c6e04a328d admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:27.002333 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92796c36-5c9f-458f-9d42-b1c6e04a328d admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:27.007071 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bfa714-a22d-4f92-aebe-82f3865a86f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:27.007673 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bfa714-a22d-4f92-aebe-82f3865a86f1 admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:27.007956 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bfa714-a22d-4f92-aebe-82f3865a86f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:27.008265 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bfa714-a22d-4f92-aebe-82f3865a86f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:27.008941 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bfa714-a22d-4f92-aebe-82f3865a86f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:27.010616 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bfa714-a22d-4f92-aebe-82f3865a86f1 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:27.021634 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1033/2030] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Sep 23 15:21:26 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 23 15:21:27.026077 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bfa714-a22d-4f92-aebe-82f3865a86f1 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:27.035969 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bfa714-a22d-4f92-aebe-82f3865a86f1 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:27.042065 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4eca82a-6f74-4c59-a889-1affac7d71b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:27.042694 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4eca82a-6f74-4c59-a889-1affac7d71b5 admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:27.043073 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4eca82a-6f74-4c59-a889-1affac7d71b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:27.043462 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4eca82a-6f74-4c59-a889-1affac7d71b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:27.043559 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:21:27.044294 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4eca82a-6f74-4c59-a889-1affac7d71b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:27.046034 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4eca82a-6f74-4c59-a889-1affac7d71b5 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:27.057725 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1034/2031] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Sep 23 15:21:27 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 23 15:21:27.080105 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4eca82a-6f74-4c59-a889-1affac7d71b5 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:27.094561 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4eca82a-6f74-4c59-a889-1affac7d71b5 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:27.099297 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a92807f-5203-4a7d-a74e-4225694630f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:27.099969 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a92807f-5203-4a7d-a74e-4225694630f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:27.100287 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a92807f-5203-4a7d-a74e-4225694630f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:27.100764 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a92807f-5203-4a7d-a74e-4225694630f0 admin admin] PATH_INFO: `/v3/projects/4b8b72a4f25c48a1888e2c9c2ebd0b2f/users/b6e428f39cac436b815804edc8ea1bab/roles/af257be2dee54d6b942f58650196582c` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:27.117460 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a92807f-5203-4a7d-a74e-4225694630f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b8b72a4f25c48a1888e2c9c2ebd0b2f, user_id=b6e428f39cac436b815804edc8ea1bab, role_id=af257be2dee54d6b942f58650196582c)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:27.121286 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a92807f-5203-4a7d-a74e-4225694630f0 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:27.270616 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-72ec72a2-7fef-4090-9a1b-52de8138b2f5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5c857ee2049543479944da34eda79b5c'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:27.283641 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-72ec72a2-7fef-4090-9a1b-52de8138b2f5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5c857ee2049543479944da34eda79b5c'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:27.292002 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-72ec72a2-7fef-4090-9a1b-52de8138b2f5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5c857ee2049543479944da34eda79b5c'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:27.308327 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1035/2032] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Sep 23 15:21:27 2022] PUT /identity/v3/projects/4b8b72a4f25c48a1888e2c9c2ebd0b2f/users/b6e428f39cac436b815804edc8ea1bab/roles/af257be2dee54d6b942f58650196582c => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:21:27.320698 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5c0d36-8afe-4f02-8b84-ba94bb2f88ae None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:27.320988 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5c0d36-8afe-4f02-8b84-ba94bb2f88ae None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:27.321281 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5c0d36-8afe-4f02-8b84-ba94bb2f88ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:27.351846 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-ca5c0d36-8afe-4f02-8b84-ba94bb2f88ae None None] MFA Rules not processed for user `b6e428f39cac436b815804edc8ea1bab`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:21:27.353094 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-ca5c0d36-8afe-4f02-8b84-ba94bb2f88ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:27.405643 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-72ec72a2-7fef-4090-9a1b-52de8138b2f5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5c857ee2049543479944da34eda79b5c'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:27.406300 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-72ec72a2-7fef-4090-9a1b-52de8138b2f5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5c857ee2049543479944da34eda79b5c'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:27.412835 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 998/2033] 10.222.0.55 () {58 vars in 1224 bytes} [Fri Sep 23 15:21:26 2022] DELETE /identity/v3/users/5c857ee2049543479944da34eda79b5c => generated 0 bytes in 495 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:21:27.417181 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1036/2034] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:21:27 2022] POST /identity/v3/auth/tokens => generated 3284 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 15:21:27.417865 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502cf8c2-a8c9-4a37-baa3-efe80af6308e None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:27.417865 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502cf8c2-a8c9-4a37-baa3-efe80af6308e None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:27.417865 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502cf8c2-a8c9-4a37-baa3-efe80af6308e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:27.421351 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5c0d36-8afe-4f02-8b84-ba94bb2f88ae None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:27.437226 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-502cf8c2-a8c9-4a37-baa3-efe80af6308e None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:21:27.438555 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-502cf8c2-a8c9-4a37-baa3-efe80af6308e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:27.488584 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5c0d36-8afe-4f02-8b84-ba94bb2f88ae None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:27.512456 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 999/2035] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:21:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 15:21:27.522643 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502cf8c2-a8c9-4a37-baa3-efe80af6308e None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:27.524092 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-502cf8c2-a8c9-4a37-baa3-efe80af6308e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:27.554531 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef30bdb2-9bf3-4d38-91d2-d1ed86d9739e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:27.555636 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef30bdb2-9bf3-4d38-91d2-d1ed86d9739e admin admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:27.555636 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef30bdb2-9bf3-4d38-91d2-d1ed86d9739e admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:27.555814 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef30bdb2-9bf3-4d38-91d2-d1ed86d9739e admin admin] PATH_INFO: `/v3/projects` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:27.556640 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef30bdb2-9bf3-4d38-91d2-d1ed86d9739e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:27.560454 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef30bdb2-9bf3-4d38-91d2-d1ed86d9739e admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:27.603473 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502cf8c2-a8c9-4a37-baa3-efe80af6308e None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:27.607762 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d618f1-258b-4b31-b5f9-941f96937960 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd259178570864ffeb936ee10dfec6b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:27.609115 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d618f1-258b-4b31-b5f9-941f96937960 service neutron] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:27.609115 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d618f1-258b-4b31-b5f9-941f96937960 service neutron] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:27.609115 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d618f1-258b-4b31-b5f9-941f96937960 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:27.621375 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d618f1-258b-4b31-b5f9-941f96937960 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:27.623197 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d618f1-258b-4b31-b5f9-941f96937960 service neutron] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:27.634403 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1000/2036] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:21:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:21:27.660718 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1037/2037] 10.222.0.55 () {60 vars in 1153 bytes} [Fri Sep 23 15:21:27 2022] POST /identity/v3/projects => generated 405 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 23 15:21:27.664652 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d618f1-258b-4b31-b5f9-941f96937960 service neutron] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:27.666169 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-06d618f1-258b-4b31-b5f9-941f96937960 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:27.679588 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d618f1-258b-4b31-b5f9-941f96937960 service neutron] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:27.738281 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a7438d-1021-4b90-bb18-d5c8aa608daa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:27.738895 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a7438d-1021-4b90-bb18-d5c8aa608daa admin admin] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:27.739174 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a7438d-1021-4b90-bb18-d5c8aa608daa admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:27.739447 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a7438d-1021-4b90-bb18-d5c8aa608daa admin admin] PATH_INFO: `/v3/users` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:27.740133 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a7438d-1021-4b90-bb18-d5c8aa608daa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:27.742060 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a7438d-1021-4b90-bb18-d5c8aa608daa admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:27.879148 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1001/2038] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Sep 23 15:21:27 2022] POST /identity/v3/users => generated 309 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:21:27.880653 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef30bdb2-9bf3-4d38-91d2-d1ed86d9739e admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:27.892500 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef30bdb2-9bf3-4d38-91d2-d1ed86d9739e admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:27.897788 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86d52cb-d779-493b-bee5-2e5a3834b7c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:27.898407 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86d52cb-d779-493b-bee5-2e5a3834b7c4 admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:27.898687 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86d52cb-d779-493b-bee5-2e5a3834b7c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:27.898957 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86d52cb-d779-493b-bee5-2e5a3834b7c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:27.899630 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86d52cb-d779-493b-bee5-2e5a3834b7c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:27.901840 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86d52cb-d779-493b-bee5-2e5a3834b7c4 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:27.912378 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1038/2039] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Sep 23 15:21:27 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:21:27.916261 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a7438d-1021-4b90-bb18-d5c8aa608daa admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:27.928451 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a7438d-1021-4b90-bb18-d5c8aa608daa admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:27.933660 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ffe869-f927-485d-bcb8-17422c34c9ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:27.939751 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ffe869-f927-485d-bcb8-17422c34c9ab admin admin] REQUEST_METHOD: `PUT` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:27.939751 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ffe869-f927-485d-bcb8-17422c34c9ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:27.939751 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ffe869-f927-485d-bcb8-17422c34c9ab admin admin] PATH_INFO: `/v3/projects/5248efd9c5a942ca82c29904a73bb5f9/users/e174f90eb34b413cb3f21851f73050da/roles/8e2919e79df040e8926f21607818b58c` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:27.950748 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ffe869-f927-485d-bcb8-17422c34c9ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=5248efd9c5a942ca82c29904a73bb5f9, user_id=e174f90eb34b413cb3f21851f73050da, role_id=8e2919e79df040e8926f21607818b58c)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:27.953835 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ffe869-f927-485d-bcb8-17422c34c9ab admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:28.053230 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1002/2040] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Sep 23 15:21:27 2022] PUT /identity/v3/projects/5248efd9c5a942ca82c29904a73bb5f9/users/e174f90eb34b413cb3f21851f73050da/roles/8e2919e79df040e8926f21607818b58c => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 15:21:28.057054 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86d52cb-d779-493b-bee5-2e5a3834b7c4 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:28.070240 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86d52cb-d779-493b-bee5-2e5a3834b7c4 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:28.131393 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9134c213-c64c-4fab-8620-baf7a45595f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:28.132065 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9134c213-c64c-4fab-8620-baf7a45595f4 admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:28.132350 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9134c213-c64c-4fab-8620-baf7a45595f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:28.139257 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9134c213-c64c-4fab-8620-baf7a45595f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:28.139257 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9134c213-c64c-4fab-8620-baf7a45595f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:28.139257 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9134c213-c64c-4fab-8620-baf7a45595f4 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:28.146749 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1039/2041] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Sep 23 15:21:28 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:21:28.151848 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ffe869-f927-485d-bcb8-17422c34c9ab admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:28.163435 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ffe869-f927-485d-bcb8-17422c34c9ab admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:28.172562 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d9e100-df19-47f9-8d77-7c24d90f389d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:28.173716 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d9e100-df19-47f9-8d77-7c24d90f389d admin admin] REQUEST_METHOD: `PUT` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:28.173716 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d9e100-df19-47f9-8d77-7c24d90f389d admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:28.173901 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d9e100-df19-47f9-8d77-7c24d90f389d admin admin] PATH_INFO: `/v3/projects/5248efd9c5a942ca82c29904a73bb5f9/users/e174f90eb34b413cb3f21851f73050da/roles/8e2919e79df040e8926f21607818b58c` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:28.175723 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d9e100-df19-47f9-8d77-7c24d90f389d admin admin] RBAC: Authorizing `identity:create_grant(project_id=5248efd9c5a942ca82c29904a73bb5f9, user_id=e174f90eb34b413cb3f21851f73050da, role_id=8e2919e79df040e8926f21607818b58c)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:28.177088 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9134c213-c64c-4fab-8620-baf7a45595f4 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:28.178532 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d9e100-df19-47f9-8d77-7c24d90f389d admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:28.179125 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-9134c213-c64c-4fab-8620-baf7a45595f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:28.190039 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1003/2042] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Sep 23 15:21:28 2022] PUT /identity/v3/projects/5248efd9c5a942ca82c29904a73bb5f9/users/e174f90eb34b413cb3f21851f73050da/roles/8e2919e79df040e8926f21607818b58c => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:21:28.195716 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9134c213-c64c-4fab-8620-baf7a45595f4 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:28.201247 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51463cdc-e889-400d-ae66-85d4a3739d44 None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:28.201507 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51463cdc-e889-400d-ae66-85d4a3739d44 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:28.208419 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51463cdc-e889-400d-ae66-85d4a3739d44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:28.236623 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-51463cdc-e889-400d-ae66-85d4a3739d44 None None] MFA Rules not processed for user `e174f90eb34b413cb3f21851f73050da`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:21:28.238494 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-51463cdc-e889-400d-ae66-85d4a3739d44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:28.264093 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67975f4-c5a3-4e2b-83f7-4b1d12b64ead admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:28.264746 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67975f4-c5a3-4e2b-83f7-4b1d12b64ead admin admin] REQUEST_METHOD: `DELETE` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:28.265030 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67975f4-c5a3-4e2b-83f7-4b1d12b64ead admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:28.265310 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67975f4-c5a3-4e2b-83f7-4b1d12b64ead admin admin] PATH_INFO: `/v3/projects/f8c92ebecfe64b8f96aeac894f6947e0` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:28.266467 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67975f4-c5a3-4e2b-83f7-4b1d12b64ead admin admin] RBAC: Authorizing `identity:delete_project(project_id=f8c92ebecfe64b8f96aeac894f6947e0)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:28.268421 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67975f4-c5a3-4e2b-83f7-4b1d12b64ead admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:28.329455 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1004/2043] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:21:28 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:21:28.339379 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:21:28.345606 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51463cdc-e889-400d-ae66-85d4a3739d44 None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:28.347151 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-51463cdc-e889-400d-ae66-85d4a3739d44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:28.415600 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51463cdc-e889-400d-ae66-85d4a3739d44 None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:28.425068 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-a67975f4-c5a3-4e2b-83f7-4b1d12b64ead admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f8c92ebecfe64b8f96aeac894f6947e0', 'request_id': 'req-a67975f4-c5a3-4e2b-83f7-4b1d12b64ead'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:28.432317 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-a67975f4-c5a3-4e2b-83f7-4b1d12b64ead admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f8c92ebecfe64b8f96aeac894f6947e0', 'request_id': 'req-a67975f4-c5a3-4e2b-83f7-4b1d12b64ead'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:28.438072 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-a67975f4-c5a3-4e2b-83f7-4b1d12b64ead admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f8c92ebecfe64b8f96aeac894f6947e0', 'request_id': 'req-a67975f4-c5a3-4e2b-83f7-4b1d12b64ead'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:28.467711 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aeee479-6ccc-451d-90b5-cad2087f6910 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feca6278be904ef8b0dc580a67e477f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:28.468367 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aeee479-6ccc-451d-90b5-cad2087f6910 service cinder] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:28.468649 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aeee479-6ccc-451d-90b5-cad2087f6910 service cinder] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:28.468923 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aeee479-6ccc-451d-90b5-cad2087f6910 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:28.532555 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aeee479-6ccc-451d-90b5-cad2087f6910 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:28.534178 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aeee479-6ccc-451d-90b5-cad2087f6910 service cinder] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:28.536644 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-a67975f4-c5a3-4e2b-83f7-4b1d12b64ead admin admin] The token cache is being invalidate because project f8c92ebecfe64b8f96aeac894f6947e0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81994) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 15:21:28.536970 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-a67975f4-c5a3-4e2b-83f7-4b1d12b64ead admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:28.537637 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-a67975f4-c5a3-4e2b-83f7-4b1d12b64ead admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:28.539360 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1040/2044] 10.222.0.55 () {58 vars in 1233 bytes} [Fri Sep 23 15:21:28 2022] DELETE /identity/v3/projects/f8c92ebecfe64b8f96aeac894f6947e0 => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 15:21:28.544672 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17eb42f-3edd-4acf-8b02-e3eecedb8cc5 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:28.545024 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17eb42f-3edd-4acf-8b02-e3eecedb8cc5 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:28.545445 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17eb42f-3edd-4acf-8b02-e3eecedb8cc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:28.547268 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1005/2045] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:21:28 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 23 15:21:28.566538 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-f17eb42f-3edd-4acf-8b02-e3eecedb8cc5 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:21:28.567865 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-f17eb42f-3edd-4acf-8b02-e3eecedb8cc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:28.640088 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1041/2046] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:21:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 15:21:28.649075 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89214bc6-5d1a-4148-be3e-080421f25e89 None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:28.649363 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89214bc6-5d1a-4148-be3e-080421f25e89 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:28.649667 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89214bc6-5d1a-4148-be3e-080421f25e89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:28.670279 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-89214bc6-5d1a-4148-be3e-080421f25e89 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:21:28.671393 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-89214bc6-5d1a-4148-be3e-080421f25e89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:28.680105 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1006/2047] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:21:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:21:28.687917 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17eb42f-3edd-4acf-8b02-e3eecedb8cc5 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:28.694982 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17eb42f-3edd-4acf-8b02-e3eecedb8cc5 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:28.701024 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca49f8a1-fddc-43a6-99f8-a86d3cf9b69e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:28.706440 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca49f8a1-fddc-43a6-99f8-a86d3cf9b69e admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:28.706440 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca49f8a1-fddc-43a6-99f8-a86d3cf9b69e admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:28.706440 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca49f8a1-fddc-43a6-99f8-a86d3cf9b69e admin admin] PATH_INFO: `/v3/domains` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:28.706440 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca49f8a1-fddc-43a6-99f8-a86d3cf9b69e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:21:28.706440 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca49f8a1-fddc-43a6-99f8-a86d3cf9b69e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:28.706440 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca49f8a1-fddc-43a6-99f8-a86d3cf9b69e admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:28.718935 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1042/2048] 10.222.0.55 () {58 vars in 1153 bytes} [Fri Sep 23 15:21:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:21:28.722495 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89214bc6-5d1a-4148-be3e-080421f25e89 None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:28.732745 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89214bc6-5d1a-4148-be3e-080421f25e89 None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:28.737612 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2f77a6-72b5-48be-bb65-3eccc608914a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:28.738224 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2f77a6-72b5-48be-bb65-3eccc608914a admin admin] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:28.738500 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2f77a6-72b5-48be-bb65-3eccc608914a admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:28.738772 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2f77a6-72b5-48be-bb65-3eccc608914a admin admin] PATH_INFO: `/v3/projects` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:28.739679 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2f77a6-72b5-48be-bb65-3eccc608914a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:28.745224 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2f77a6-72b5-48be-bb65-3eccc608914a admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:28.871925 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1007/2049] 10.222.0.55 () {60 vars in 1153 bytes} [Fri Sep 23 15:21:28 2022] POST /identity/v3/projects => generated 389 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:21:28.876209 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca49f8a1-fddc-43a6-99f8-a86d3cf9b69e admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:28.886478 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca49f8a1-fddc-43a6-99f8-a86d3cf9b69e admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:28.944805 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c690e52-0246-42f5-9d66-29e7161a84f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:28.945417 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c690e52-0246-42f5-9d66-29e7161a84f9 admin admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:28.945699 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c690e52-0246-42f5-9d66-29e7161a84f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:28.945978 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c690e52-0246-42f5-9d66-29e7161a84f9 admin admin] PATH_INFO: `/v3/users` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:28.946655 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c690e52-0246-42f5-9d66-29e7161a84f9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:28.949072 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c690e52-0246-42f5-9d66-29e7161a84f9 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:29.045861 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2f77a6-72b5-48be-bb65-3eccc608914a admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:29.047575 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-ec2f77a6-72b5-48be-bb65-3eccc608914a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:29.115944 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2f77a6-72b5-48be-bb65-3eccc608914a admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:29.120886 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f1b3e8-00f5-40d5-8b3e-d57c3742fa2f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feca6278be904ef8b0dc580a67e477f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:29.121306 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f1b3e8-00f5-40d5-8b3e-d57c3742fa2f service cinder] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:29.124587 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f1b3e8-00f5-40d5-8b3e-d57c3742fa2f service cinder] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:29.124703 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1043/2050] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Sep 23 15:21:28 2022] POST /identity/v3/users => generated 295 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 23 15:21:29.124801 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f1b3e8-00f5-40d5-8b3e-d57c3742fa2f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:29.124801 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-46f1b3e8-00f5-40d5-8b3e-d57c3742fa2f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:29.126982 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c690e52-0246-42f5-9d66-29e7161a84f9 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:29.153900 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c690e52-0246-42f5-9d66-29e7161a84f9 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:29.153900 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57db2d2-15cf-4e97-bffc-e5a86730ed51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:29.153900 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57db2d2-15cf-4e97-bffc-e5a86730ed51 admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:29.153900 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57db2d2-15cf-4e97-bffc-e5a86730ed51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:29.154266 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57db2d2-15cf-4e97-bffc-e5a86730ed51 admin admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:29.154266 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57db2d2-15cf-4e97-bffc-e5a86730ed51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:29.155659 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57db2d2-15cf-4e97-bffc-e5a86730ed51 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:29.171826 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1044/2051] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Sep 23 15:21:29 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:21:29.173120 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57db2d2-15cf-4e97-bffc-e5a86730ed51 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:29.183963 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57db2d2-15cf-4e97-bffc-e5a86730ed51 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:29.189051 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e73741e-24d4-457c-966b-1664a9ed3b42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:29.189719 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e73741e-24d4-457c-966b-1664a9ed3b42 admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:29.189998 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e73741e-24d4-457c-966b-1664a9ed3b42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:29.190265 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e73741e-24d4-457c-966b-1664a9ed3b42 admin admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:29.191025 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e73741e-24d4-457c-966b-1664a9ed3b42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:29.192596 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e73741e-24d4-457c-966b-1664a9ed3b42 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:29.195847 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f1b3e8-00f5-40d5-8b3e-d57c3742fa2f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:29.197545 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f1b3e8-00f5-40d5-8b3e-d57c3742fa2f service cinder] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:29.207108 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1045/2052] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Sep 23 15:21:29 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 23 15:21:29.207492 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1008/2053] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:21:29 2022] GET /identity/v3/auth/tokens => generated 3284 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 23 15:21:29.207966 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e73741e-24d4-457c-966b-1664a9ed3b42 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:29.218634 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e73741e-24d4-457c-966b-1664a9ed3b42 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:29.223418 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ede850-e13a-45c2-8c3e-65457a8c515f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:29.224123 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ede850-e13a-45c2-8c3e-65457a8c515f admin admin] REQUEST_METHOD: `PUT` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:29.224483 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ede850-e13a-45c2-8c3e-65457a8c515f admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:29.224780 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ede850-e13a-45c2-8c3e-65457a8c515f admin admin] PATH_INFO: `/v3/projects/f9ad8d24654c45b79bcb0bc4576476e8/users/a0a92f89ac734b84b52ccf19e7ab9139/roles/af257be2dee54d6b942f58650196582c` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:29.241147 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ede850-e13a-45c2-8c3e-65457a8c515f admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9ad8d24654c45b79bcb0bc4576476e8, user_id=a0a92f89ac734b84b52ccf19e7ab9139, role_id=af257be2dee54d6b942f58650196582c)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:29.243892 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ede850-e13a-45c2-8c3e-65457a8c515f admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:29.337863 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1046/2054] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Sep 23 15:21:29 2022] PUT /identity/v3/projects/f9ad8d24654c45b79bcb0bc4576476e8/users/a0a92f89ac734b84b52ccf19e7ab9139/roles/af257be2dee54d6b942f58650196582c => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:21:29.349613 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f09919a-f9fa-48ac-a9fd-7f110848169f None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:29.349958 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f09919a-f9fa-48ac-a9fd-7f110848169f None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:29.350262 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f09919a-f9fa-48ac-a9fd-7f110848169f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:29.382696 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-9f09919a-f9fa-48ac-a9fd-7f110848169f None None] MFA Rules not processed for user `a0a92f89ac734b84b52ccf19e7ab9139`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:21:29.383936 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-9f09919a-f9fa-48ac-a9fd-7f110848169f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:29.451196 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1009/2055] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:21:29 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 15:21:29.740314 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ede850-e13a-45c2-8c3e-65457a8c515f admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:29.742057 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-89ede850-e13a-45c2-8c3e-65457a8c515f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:29.802053 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ede850-e13a-45c2-8c3e-65457a8c515f admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:29.806731 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5849c2c5-737c-48ff-8f75-c13add3d94b5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feca6278be904ef8b0dc580a67e477f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:29.806731 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5849c2c5-737c-48ff-8f75-c13add3d94b5 service cinder] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:29.806928 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5849c2c5-737c-48ff-8f75-c13add3d94b5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:29.807369 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5849c2c5-737c-48ff-8f75-c13add3d94b5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:29.815926 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5849c2c5-737c-48ff-8f75-c13add3d94b5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:29.817381 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5849c2c5-737c-48ff-8f75-c13add3d94b5 service cinder] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:29.827069 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1047/2056] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:21:29 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:21:29.841070 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:21:31.331933 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:21:31.988813 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:21:33.381296 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:21:34.988596 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:21:35.691611 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:21:36.233876 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:21:36.331383 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:21:37.297086 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:21:37.502774 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:21:37.995050 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:21:38.135846 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:21:39.085205 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:21:39.434059 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:21:40.483076 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:21:40.731358 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:21:42.006113 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=80,keystone:DELETE=17,keystone:INSERT=3 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:21:42.250976 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:21:43.119388 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:21:43.250888 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:DELETE=12,keystone:SELECT=83,keystone:INSERT=5 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:21:44.380280 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:21:44.502216 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:21:45.113718 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:21:45.524933 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:21:45.678901 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:21:46.181976 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:DELETE=1,keystone:SELECT=97,keystone:INSERT=2 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:21:46.284638 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:21:47.045413 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:21:47.124911 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:21:48.376432 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:21:48.448840 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:21:48.993191 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:21:49.380921 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:21:49.982705 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:21:50.856875 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:21:51.444713 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:21:51.541777 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=97,keystone:INSERT=1,keystone:DELETE=2 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:21:51.723082 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:21:51.850414 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:21:51.890496 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:21:52.050531 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:21:52.204689 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:21:52.381610 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:21:52.416145 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:21:52.478735 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:21:52.644213 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:21:52.678620 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:21:53.123387 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:21:53.345537 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=80,keystone:DELETE=17,keystone:INSERT=3 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:21:53.771926 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:21:54.381648 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:21:54.717893 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f09919a-f9fa-48ac-a9fd-7f110848169f None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:54.719637 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-9f09919a-f9fa-48ac-a9fd-7f110848169f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:54.823503 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f09919a-f9fa-48ac-a9fd-7f110848169f None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:54.828703 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d1be63-ed3b-4c2c-ae39-ab0bb44b1388 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:54.829551 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d1be63-ed3b-4c2c-ae39-ab0bb44b1388 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:54.829894 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d1be63-ed3b-4c2c-ae39-ab0bb44b1388 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:54.830235 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d1be63-ed3b-4c2c-ae39-ab0bb44b1388 admin admin] PATH_INFO: `/v3/users/b6e428f39cac436b815804edc8ea1bab` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:54.831616 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d1be63-ed3b-4c2c-ae39-ab0bb44b1388 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b6e428f39cac436b815804edc8ea1bab)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:54.834606 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d1be63-ed3b-4c2c-ae39-ab0bb44b1388 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:55.110506 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-05d1be63-ed3b-4c2c-ae39-ab0bb44b1388 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b6e428f39cac436b815804edc8ea1bab'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:55.127644 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-05d1be63-ed3b-4c2c-ae39-ab0bb44b1388 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b6e428f39cac436b815804edc8ea1bab'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:55.140369 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-05d1be63-ed3b-4c2c-ae39-ab0bb44b1388 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b6e428f39cac436b815804edc8ea1bab'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:55.176011 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:21:55.249750 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-05d1be63-ed3b-4c2c-ae39-ab0bb44b1388 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b6e428f39cac436b815804edc8ea1bab'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:55.250379 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-05d1be63-ed3b-4c2c-ae39-ab0bb44b1388 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b6e428f39cac436b815804edc8ea1bab'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:55.257031 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1010/2057] 10.222.0.55 () {58 vars in 1224 bytes} [Fri Sep 23 15:21:54 2022] DELETE /identity/v3/users/b6e428f39cac436b815804edc8ea1bab => generated 0 bytes in 541 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:21:55.265681 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef66212d-9448-40cc-89d6-3e2145aa0046 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:55.265970 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef66212d-9448-40cc-89d6-3e2145aa0046 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:55.266239 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef66212d-9448-40cc-89d6-3e2145aa0046 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:55.287258 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-ef66212d-9448-40cc-89d6-3e2145aa0046 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:21:55.288601 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-ef66212d-9448-40cc-89d6-3e2145aa0046 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:55.304372 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:21:55.367948 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1048/2058] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:21:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 15:21:55.378939 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d1be63-ed3b-4c2c-ae39-ab0bb44b1388 admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:55.380619 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-05d1be63-ed3b-4c2c-ae39-ab0bb44b1388 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:55.445884 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d1be63-ed3b-4c2c-ae39-ab0bb44b1388 admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:55.450874 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be00cd03-3766-404f-a755-f9db064a7dbd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd259178570864ffeb936ee10dfec6b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:55.451593 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be00cd03-3766-404f-a755-f9db064a7dbd service neutron] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:55.451995 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be00cd03-3766-404f-a755-f9db064a7dbd service neutron] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:55.452438 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be00cd03-3766-404f-a755-f9db064a7dbd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:55.462248 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be00cd03-3766-404f-a755-f9db064a7dbd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:55.463830 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be00cd03-3766-404f-a755-f9db064a7dbd service neutron] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:55.473979 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1011/2059] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:21:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:21:55.788241 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:21:55.998257 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef66212d-9448-40cc-89d6-3e2145aa0046 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:55.998257 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-ef66212d-9448-40cc-89d6-3e2145aa0046 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:56.014799 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef66212d-9448-40cc-89d6-3e2145aa0046 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:56.019836 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6643190-ecc5-4862-905e-6a9a4b8fcc65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:56.021159 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6643190-ecc5-4862-905e-6a9a4b8fcc65 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:56.021633 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6643190-ecc5-4862-905e-6a9a4b8fcc65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:56.022230 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6643190-ecc5-4862-905e-6a9a4b8fcc65 admin admin] PATH_INFO: `/v3/projects/4b8b72a4f25c48a1888e2c9c2ebd0b2f` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:56.023397 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6643190-ecc5-4862-905e-6a9a4b8fcc65 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4b8b72a4f25c48a1888e2c9c2ebd0b2f)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:56.025378 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6643190-ecc5-4862-905e-6a9a4b8fcc65 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:56.191672 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-d6643190-ecc5-4862-905e-6a9a4b8fcc65 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4b8b72a4f25c48a1888e2c9c2ebd0b2f', 'request_id': 'req-d6643190-ecc5-4862-905e-6a9a4b8fcc65'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:56.200175 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-d6643190-ecc5-4862-905e-6a9a4b8fcc65 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4b8b72a4f25c48a1888e2c9c2ebd0b2f', 'request_id': 'req-d6643190-ecc5-4862-905e-6a9a4b8fcc65'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:56.207956 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-d6643190-ecc5-4862-905e-6a9a4b8fcc65 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4b8b72a4f25c48a1888e2c9c2ebd0b2f', 'request_id': 'req-d6643190-ecc5-4862-905e-6a9a4b8fcc65'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:56.269609 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:21:56.309969 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-d6643190-ecc5-4862-905e-6a9a4b8fcc65 admin admin] The token cache is being invalidate because project 4b8b72a4f25c48a1888e2c9c2ebd0b2f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81994) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 15:21:56.310565 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-d6643190-ecc5-4862-905e-6a9a4b8fcc65 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:56.311940 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-d6643190-ecc5-4862-905e-6a9a4b8fcc65 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:56.313909 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1049/2060] 10.222.0.55 () {58 vars in 1233 bytes} [Fri Sep 23 15:21:55 2022] DELETE /identity/v3/projects/4b8b72a4f25c48a1888e2c9c2ebd0b2f => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:21:56.317946 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be00cd03-3766-404f-a755-f9db064a7dbd service neutron] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:56.330877 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be00cd03-3766-404f-a755-f9db064a7dbd service neutron] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:56.394256 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:21:56.408486 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c532f35e-661b-48f9-b6ca-2682b9628e81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:56.409365 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c532f35e-661b-48f9-b6ca-2682b9628e81 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:56.409646 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c532f35e-661b-48f9-b6ca-2682b9628e81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:56.409937 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c532f35e-661b-48f9-b6ca-2682b9628e81 admin admin] PATH_INFO: `/v3/users/e174f90eb34b413cb3f21851f73050da` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:56.411117 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c532f35e-661b-48f9-b6ca-2682b9628e81 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e174f90eb34b413cb3f21851f73050da)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:56.413098 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c532f35e-661b-48f9-b6ca-2682b9628e81 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:56.776149 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-c532f35e-661b-48f9-b6ca-2682b9628e81 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e174f90eb34b413cb3f21851f73050da'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:56.791769 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-c532f35e-661b-48f9-b6ca-2682b9628e81 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e174f90eb34b413cb3f21851f73050da'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:56.803059 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-c532f35e-661b-48f9-b6ca-2682b9628e81 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e174f90eb34b413cb3f21851f73050da'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:56.958363 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-c532f35e-661b-48f9-b6ca-2682b9628e81 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e174f90eb34b413cb3f21851f73050da'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:56.958991 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-c532f35e-661b-48f9-b6ca-2682b9628e81 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e174f90eb34b413cb3f21851f73050da'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:56.965796 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1012/2061] 10.222.0.55 () {58 vars in 1224 bytes} [Fri Sep 23 15:21:56 2022] DELETE /identity/v3/users/e174f90eb34b413cb3f21851f73050da => generated 0 bytes in 648 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:21:57.551435 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:21:57.613632 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6643190-ecc5-4862-905e-6a9a4b8fcc65 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:57.615144 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-d6643190-ecc5-4862-905e-6a9a4b8fcc65 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:57.651606 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6643190-ecc5-4862-905e-6a9a4b8fcc65 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:57.710295 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acfaebe-7296-4c94-a55a-dadd939d4de1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:57.710936 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acfaebe-7296-4c94-a55a-dadd939d4de1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:57.711321 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acfaebe-7296-4c94-a55a-dadd939d4de1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:57.711597 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acfaebe-7296-4c94-a55a-dadd939d4de1 admin admin] PATH_INFO: `/v3/projects/5248efd9c5a942ca82c29904a73bb5f9` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:57.712908 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acfaebe-7296-4c94-a55a-dadd939d4de1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5248efd9c5a942ca82c29904a73bb5f9)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:57.714811 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acfaebe-7296-4c94-a55a-dadd939d4de1 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:57.947817 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-9acfaebe-7296-4c94-a55a-dadd939d4de1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5248efd9c5a942ca82c29904a73bb5f9', 'request_id': 'req-9acfaebe-7296-4c94-a55a-dadd939d4de1'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:57.955138 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-9acfaebe-7296-4c94-a55a-dadd939d4de1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5248efd9c5a942ca82c29904a73bb5f9', 'request_id': 'req-9acfaebe-7296-4c94-a55a-dadd939d4de1'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:57.960964 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-9acfaebe-7296-4c94-a55a-dadd939d4de1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5248efd9c5a942ca82c29904a73bb5f9', 'request_id': 'req-9acfaebe-7296-4c94-a55a-dadd939d4de1'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:58.112780 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-9acfaebe-7296-4c94-a55a-dadd939d4de1 admin admin] The token cache is being invalidate because project 5248efd9c5a942ca82c29904a73bb5f9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81994) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 15:21:58.113072 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-9acfaebe-7296-4c94-a55a-dadd939d4de1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:58.113720 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-9acfaebe-7296-4c94-a55a-dadd939d4de1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:58.115212 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1050/2062] 10.222.0.55 () {58 vars in 1233 bytes} [Fri Sep 23 15:21:57 2022] DELETE /identity/v3/projects/5248efd9c5a942ca82c29904a73bb5f9 => generated 0 bytes in 502 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:21:58.120195 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f496e165-35d8-44f0-b08f-85c36a9d5e7c None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:58.120707 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f496e165-35d8-44f0-b08f-85c36a9d5e7c None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:58.120815 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f496e165-35d8-44f0-b08f-85c36a9d5e7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:58.141083 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-f496e165-35d8-44f0-b08f-85c36a9d5e7c None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:21:58.142276 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-f496e165-35d8-44f0-b08f-85c36a9d5e7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:58.210014 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1013/2063] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:21:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:21:58.231099 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be8c82c-461f-44b4-b0be-2dc3a500ee7a None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:58.231925 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be8c82c-461f-44b4-b0be-2dc3a500ee7a None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:58.233310 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be8c82c-461f-44b4-b0be-2dc3a500ee7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:58.265636 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-1be8c82c-461f-44b4-b0be-2dc3a500ee7a None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:21:58.266902 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-1be8c82c-461f-44b4-b0be-2dc3a500ee7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:58.284962 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1051/2064] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:21:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:21:58.288885 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f496e165-35d8-44f0-b08f-85c36a9d5e7c None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:58.306206 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f496e165-35d8-44f0-b08f-85c36a9d5e7c None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:58.313670 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2235c5d-b6ec-47ee-ad52-b1146cc8eca4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:58.315952 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2235c5d-b6ec-47ee-ad52-b1146cc8eca4 admin admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:58.316454 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2235c5d-b6ec-47ee-ad52-b1146cc8eca4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:58.317581 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2235c5d-b6ec-47ee-ad52-b1146cc8eca4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:58.321535 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2235c5d-b6ec-47ee-ad52-b1146cc8eca4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:21:58.322020 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2235c5d-b6ec-47ee-ad52-b1146cc8eca4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:58.324742 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2235c5d-b6ec-47ee-ad52-b1146cc8eca4 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:58.346533 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1014/2065] 10.222.0.55 () {58 vars in 1153 bytes} [Fri Sep 23 15:21:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 15:21:58.349213 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be8c82c-461f-44b4-b0be-2dc3a500ee7a None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:58.364559 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be8c82c-461f-44b4-b0be-2dc3a500ee7a None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:58.387219 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eaf7969-8d62-4abb-bef4-42743e8f1b3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:58.392276 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaf7969-8d62-4abb-bef4-42743e8f1b3b admin admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:58.392701 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaf7969-8d62-4abb-bef4-42743e8f1b3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:58.393201 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaf7969-8d62-4abb-bef4-42743e8f1b3b admin admin] PATH_INFO: `/v3/projects` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:58.394055 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eaf7969-8d62-4abb-bef4-42743e8f1b3b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:58.396143 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eaf7969-8d62-4abb-bef4-42743e8f1b3b admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:58.563015 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1052/2066] 10.222.0.55 () {60 vars in 1153 bytes} [Fri Sep 23 15:21:58 2022] POST /identity/v3/projects => generated 365 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 23 15:21:58.565966 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2235c5d-b6ec-47ee-ad52-b1146cc8eca4 admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:58.576610 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2235c5d-b6ec-47ee-ad52-b1146cc8eca4 admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:58.610064 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:21:58.640289 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb148fa3-2cbf-4f98-961f-823634e7721f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:58.640837 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb148fa3-2cbf-4f98-961f-823634e7721f admin admin] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:58.641115 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb148fa3-2cbf-4f98-961f-823634e7721f admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:58.641386 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb148fa3-2cbf-4f98-961f-823634e7721f admin admin] PATH_INFO: `/v3/users` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:58.642016 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb148fa3-2cbf-4f98-961f-823634e7721f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:58.644105 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb148fa3-2cbf-4f98-961f-823634e7721f admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:58.777122 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1015/2067] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Sep 23 15:21:58 2022] POST /identity/v3/users => generated 283 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:21:58.777266 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eaf7969-8d62-4abb-bef4-42743e8f1b3b admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:58.789433 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eaf7969-8d62-4abb-bef4-42743e8f1b3b admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:58.806264 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c355de25-e2eb-4b6b-963f-aa574fbda663 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:58.806841 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c355de25-e2eb-4b6b-963f-aa574fbda663 admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:58.807328 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c355de25-e2eb-4b6b-963f-aa574fbda663 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:58.807424 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c355de25-e2eb-4b6b-963f-aa574fbda663 admin admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:58.814740 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c355de25-e2eb-4b6b-963f-aa574fbda663 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:58.816415 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c355de25-e2eb-4b6b-963f-aa574fbda663 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:58.832472 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1053/2068] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Sep 23 15:21:58 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:21:58.838419 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb148fa3-2cbf-4f98-961f-823634e7721f admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:58.876931 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb148fa3-2cbf-4f98-961f-823634e7721f admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:58.895671 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5363415f-9ef4-4567-95b1-b06e9c18a82b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:58.897089 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5363415f-9ef4-4567-95b1-b06e9c18a82b admin admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:58.897381 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5363415f-9ef4-4567-95b1-b06e9c18a82b admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:58.897663 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5363415f-9ef4-4567-95b1-b06e9c18a82b admin admin] PATH_INFO: `/v3/roles` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:58.898441 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5363415f-9ef4-4567-95b1-b06e9c18a82b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:58.928969 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5363415f-9ef4-4567-95b1-b06e9c18a82b admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:58.947074 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:21:58.951642 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1016/2069] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Sep 23 15:21:58 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:21:58.953687 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c355de25-e2eb-4b6b-963f-aa574fbda663 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:58.973019 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c355de25-e2eb-4b6b-963f-aa574fbda663 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:58.980672 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61490a01-d873-4d00-8ce4-3cefdd5b3c57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:58.981487 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61490a01-d873-4d00-8ce4-3cefdd5b3c57 admin admin] REQUEST_METHOD: `PUT` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:58.982075 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61490a01-d873-4d00-8ce4-3cefdd5b3c57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:58.982445 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61490a01-d873-4d00-8ce4-3cefdd5b3c57 admin admin] PATH_INFO: `/v3/projects/70a4b13673f94eca9e57598dd454dd87/users/a3430eb5e7104dc2b3a1c3500614506e/roles/af257be2dee54d6b942f58650196582c` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:59.000622 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61490a01-d873-4d00-8ce4-3cefdd5b3c57 admin admin] RBAC: Authorizing `identity:create_grant(project_id=70a4b13673f94eca9e57598dd454dd87, user_id=a3430eb5e7104dc2b3a1c3500614506e, role_id=af257be2dee54d6b942f58650196582c)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:59.003738 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61490a01-d873-4d00-8ce4-3cefdd5b3c57 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:59.095334 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5363415f-9ef4-4567-95b1-b06e9c18a82b admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:59.097256 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-5363415f-9ef4-4567-95b1-b06e9c18a82b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:59.112524 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5363415f-9ef4-4567-95b1-b06e9c18a82b admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:59.117324 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23156c58-605f-48ee-8743-2f108f44b264 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:59.117966 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23156c58-605f-48ee-8743-2f108f44b264 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:59.118249 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23156c58-605f-48ee-8743-2f108f44b264 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:59.118524 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23156c58-605f-48ee-8743-2f108f44b264 admin admin] PATH_INFO: `/v3/users/69dc5c52e36841f5aad028fc6a4b3daf` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:59.119648 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23156c58-605f-48ee-8743-2f108f44b264 admin admin] RBAC: Authorizing `identity:delete_user(user_id=69dc5c52e36841f5aad028fc6a4b3daf)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:59.121969 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23156c58-605f-48ee-8743-2f108f44b264 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:59.129909 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1054/2070] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Sep 23 15:21:58 2022] PUT /identity/v3/projects/70a4b13673f94eca9e57598dd454dd87/users/a3430eb5e7104dc2b3a1c3500614506e/roles/af257be2dee54d6b942f58650196582c => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:21:59.141406 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e17d3e-4e13-49e7-867e-af8e56cb4b41 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:59.141696 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e17d3e-4e13-49e7-867e-af8e56cb4b41 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:59.141973 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e17d3e-4e13-49e7-867e-af8e56cb4b41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:59.170414 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-69e17d3e-4e13-49e7-867e-af8e56cb4b41 None None] MFA Rules not processed for user `a3430eb5e7104dc2b3a1c3500614506e`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:21:59.171662 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-69e17d3e-4e13-49e7-867e-af8e56cb4b41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:59.242423 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1055/2071] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:21:59 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 15:21:59.247771 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e17d3e-4e13-49e7-867e-af8e56cb4b41 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:59.313444 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e17d3e-4e13-49e7-867e-af8e56cb4b41 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:59.318089 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960e72e7-28df-40d5-9ba3-d621e0e36ffc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:59.318704 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960e72e7-28df-40d5-9ba3-d621e0e36ffc admin admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:59.318983 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960e72e7-28df-40d5-9ba3-d621e0e36ffc admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:59.319255 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960e72e7-28df-40d5-9ba3-d621e0e36ffc admin admin] PATH_INFO: `/v3/projects` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:59.319947 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960e72e7-28df-40d5-9ba3-d621e0e36ffc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:59.321947 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960e72e7-28df-40d5-9ba3-d621e0e36ffc admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:59.360350 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-23156c58-605f-48ee-8743-2f108f44b264 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '69dc5c52e36841f5aad028fc6a4b3daf'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:59.373204 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-23156c58-605f-48ee-8743-2f108f44b264 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '69dc5c52e36841f5aad028fc6a4b3daf'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:59.380202 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-23156c58-605f-48ee-8743-2f108f44b264 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '69dc5c52e36841f5aad028fc6a4b3daf'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:59.476496 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-23156c58-605f-48ee-8743-2f108f44b264 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '69dc5c52e36841f5aad028fc6a4b3daf'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:59.477207 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-23156c58-605f-48ee-8743-2f108f44b264 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '69dc5c52e36841f5aad028fc6a4b3daf'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:59.484254 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1017/2072] 10.222.0.55 () {58 vars in 1224 bytes} [Fri Sep 23 15:21:59 2022] DELETE /identity/v3/users/69dc5c52e36841f5aad028fc6a4b3daf => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 15:21:59.491614 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1056/2073] 10.222.0.55 () {60 vars in 1153 bytes} [Fri Sep 23 15:21:59 2022] POST /identity/v3/projects => generated 367 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 23 15:21:59.492422 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23156c58-605f-48ee-8743-2f108f44b264 admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:59.493887 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-23156c58-605f-48ee-8743-2f108f44b264 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:59.565592 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23156c58-605f-48ee-8743-2f108f44b264 admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:59.623616 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb353c61-f88b-4ae5-b33e-3a394b0e3ba7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:59.624284 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb353c61-f88b-4ae5-b33e-3a394b0e3ba7 admin admin] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:59.624567 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb353c61-f88b-4ae5-b33e-3a394b0e3ba7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:59.624840 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb353c61-f88b-4ae5-b33e-3a394b0e3ba7 admin admin] PATH_INFO: `/v3/users` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:59.625490 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb353c61-f88b-4ae5-b33e-3a394b0e3ba7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:59.627549 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb353c61-f88b-4ae5-b33e-3a394b0e3ba7 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:59.880670 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960e72e7-28df-40d5-9ba3-d621e0e36ffc admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:59.890138 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-960e72e7-28df-40d5-9ba3-d621e0e36ffc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:59.901061 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1018/2074] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Sep 23 15:21:59 2022] POST /identity/v3/users => generated 290 bytes in 409 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:21:59.902091 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb353c61-f88b-4ae5-b33e-3a394b0e3ba7 admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:59.908896 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960e72e7-28df-40d5-9ba3-d621e0e36ffc admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:59.916539 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb353c61-f88b-4ae5-b33e-3a394b0e3ba7 admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:59.924485 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe90542b-f4f5-4590-a6ed-bc97d621bf9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:59.924485 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe90542b-f4f5-4590-a6ed-bc97d621bf9e admin admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:59.924485 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe90542b-f4f5-4590-a6ed-bc97d621bf9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:59.924485 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe90542b-f4f5-4590-a6ed-bc97d621bf9e admin admin] PATH_INFO: `/v3/roles` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:59.924975 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2ace8e-c920-4e3c-b066-71b0d35fc63b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:59.925226 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe90542b-f4f5-4590-a6ed-bc97d621bf9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:59.925740 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2ace8e-c920-4e3c-b066-71b0d35fc63b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:59.926124 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2ace8e-c920-4e3c-b066-71b0d35fc63b admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:59.926527 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2ace8e-c920-4e3c-b066-71b0d35fc63b admin admin] PATH_INFO: `/v3/projects/df328392b4f54d0c92df32df0241e95d` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:59.927282 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe90542b-f4f5-4590-a6ed-bc97d621bf9e admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:59.927922 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2ace8e-c920-4e3c-b066-71b0d35fc63b admin admin] RBAC: Authorizing `identity:delete_project(project_id=df328392b4f54d0c92df32df0241e95d)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:59.930037 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2ace8e-c920-4e3c-b066-71b0d35fc63b admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:59.939008 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1019/2075] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Sep 23 15:21:59 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:21:59.943065 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe90542b-f4f5-4590-a6ed-bc97d621bf9e admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:59.953377 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe90542b-f4f5-4590-a6ed-bc97d621bf9e admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:59.957937 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e610ec-bd5f-47e3-a116-32a28634e94a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:59.958588 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e610ec-bd5f-47e3-a116-32a28634e94a admin admin] REQUEST_METHOD: `PUT` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:59.958866 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e610ec-bd5f-47e3-a116-32a28634e94a admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:59.959143 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e610ec-bd5f-47e3-a116-32a28634e94a admin admin] PATH_INFO: `/v3/projects/ff892905862b43eabe4699507af00b6c/users/f2c57745fb374059a028a24d2ee57576/roles/8e2919e79df040e8926f21607818b58c` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:59.978685 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e610ec-bd5f-47e3-a116-32a28634e94a admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff892905862b43eabe4699507af00b6c, user_id=f2c57745fb374059a028a24d2ee57576, role_id=8e2919e79df040e8926f21607818b58c)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:59.984767 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e610ec-bd5f-47e3-a116-32a28634e94a admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:22:00.091590 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1020/2076] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Sep 23 15:21:59 2022] PUT /identity/v3/projects/ff892905862b43eabe4699507af00b6c/users/f2c57745fb374059a028a24d2ee57576/roles/8e2919e79df040e8926f21607818b58c => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 15:22:00.095816 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e610ec-bd5f-47e3-a116-32a28634e94a admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:22:00.100509 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-dd2ace8e-c920-4e3c-b066-71b0d35fc63b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'df328392b4f54d0c92df32df0241e95d', 'request_id': 'req-dd2ace8e-c920-4e3c-b066-71b0d35fc63b'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:22:00.108828 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-dd2ace8e-c920-4e3c-b066-71b0d35fc63b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'df328392b4f54d0c92df32df0241e95d', 'request_id': 'req-dd2ace8e-c920-4e3c-b066-71b0d35fc63b'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:22:00.115474 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-dd2ace8e-c920-4e3c-b066-71b0d35fc63b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'df328392b4f54d0c92df32df0241e95d', 'request_id': 'req-dd2ace8e-c920-4e3c-b066-71b0d35fc63b'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:22:00.164209 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e610ec-bd5f-47e3-a116-32a28634e94a admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:22:00.169195 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8feac700-0a5d-4f28-9e48-f8df99a5fc75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:22:00.170065 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8feac700-0a5d-4f28-9e48-f8df99a5fc75 admin admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:00.170383 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8feac700-0a5d-4f28-9e48-f8df99a5fc75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:00.170659 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8feac700-0a5d-4f28-9e48-f8df99a5fc75 admin admin] PATH_INFO: `/v3/roles` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:00.171335 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8feac700-0a5d-4f28-9e48-f8df99a5fc75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:22:00.172960 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8feac700-0a5d-4f28-9e48-f8df99a5fc75 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:22:00.184368 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1021/2077] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Sep 23 15:22:00 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:22:00.189426 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8feac700-0a5d-4f28-9e48-f8df99a5fc75 admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:22:00.204958 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8feac700-0a5d-4f28-9e48-f8df99a5fc75 admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:22:00.211259 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb08958-9680-4514-ae28-95a60b64665d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:22:00.211259 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb08958-9680-4514-ae28-95a60b64665d admin admin] REQUEST_METHOD: `PUT` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:00.211480 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb08958-9680-4514-ae28-95a60b64665d admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:00.211709 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb08958-9680-4514-ae28-95a60b64665d admin admin] PATH_INFO: `/v3/projects/ff892905862b43eabe4699507af00b6c/users/f2c57745fb374059a028a24d2ee57576/roles/8e2919e79df040e8926f21607818b58c` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:00.214199 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb08958-9680-4514-ae28-95a60b64665d admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff892905862b43eabe4699507af00b6c, user_id=f2c57745fb374059a028a24d2ee57576, role_id=8e2919e79df040e8926f21607818b58c)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:22:00.217961 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb08958-9680-4514-ae28-95a60b64665d admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:22:00.220305 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-dd2ace8e-c920-4e3c-b066-71b0d35fc63b admin admin] The token cache is being invalidate because project df328392b4f54d0c92df32df0241e95d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81994) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 15:22:00.220919 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-dd2ace8e-c920-4e3c-b066-71b0d35fc63b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:22:00.221909 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-dd2ace8e-c920-4e3c-b066-71b0d35fc63b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:22:00.234188 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1057/2078] 10.222.0.55 () {58 vars in 1233 bytes} [Fri Sep 23 15:21:59 2022] DELETE /identity/v3/projects/df328392b4f54d0c92df32df0241e95d => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 15:22:00.240642 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1022/2079] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Sep 23 15:22:00 2022] PUT /identity/v3/projects/ff892905862b43eabe4699507af00b6c/users/f2c57745fb374059a028a24d2ee57576/roles/8e2919e79df040e8926f21607818b58c => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 15:22:00.252633 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb386fa-554d-43e1-97c1-dc4559998bb9 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:00.253203 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb386fa-554d-43e1-97c1-dc4559998bb9 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:00.253769 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb386fa-554d-43e1-97c1-dc4559998bb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:00.261169 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31a2c19-ff20-489c-8bc7-aa8fff448edd None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:00.263230 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31a2c19-ff20-489c-8bc7-aa8fff448edd None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:00.265428 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31a2c19-ff20-489c-8bc7-aa8fff448edd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:00.287516 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-0cb386fa-554d-43e1-97c1-dc4559998bb9 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:22:00.289403 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-0cb386fa-554d-43e1-97c1-dc4559998bb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:22:00.313001 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-e31a2c19-ff20-489c-8bc7-aa8fff448edd None None] MFA Rules not processed for user `f2c57745fb374059a028a24d2ee57576`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:22:00.314934 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-e31a2c19-ff20-489c-8bc7-aa8fff448edd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:22:00.376849 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1058/2080] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:22:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:22:00.384491 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac7fd19-30ac-4bc8-aeab-f1d6f8c53e25 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:00.385275 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac7fd19-30ac-4bc8-aeab-f1d6f8c53e25 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:00.385693 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac7fd19-30ac-4bc8-aeab-f1d6f8c53e25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:00.414293 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-bac7fd19-30ac-4bc8-aeab-f1d6f8c53e25 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:22:00.414399 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1023/2081] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:22:00 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:22:00.414498 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-bac7fd19-30ac-4bc8-aeab-f1d6f8c53e25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:22:00.429214 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1059/2082] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:22:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 15:22:00.430124 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31a2c19-ff20-489c-8bc7-aa8fff448edd None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:22:00.444764 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31a2c19-ff20-489c-8bc7-aa8fff448edd None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:22:00.447864 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac7fd19-30ac-4bc8-aeab-f1d6f8c53e25 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:22:00.449501 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-bac7fd19-30ac-4bc8-aeab-f1d6f8c53e25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:22:00.450719 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9719136-debd-49fe-b1a6-5d6acdd67fb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:22:00.451907 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9719136-debd-49fe-b1a6-5d6acdd67fb3 admin admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:00.452222 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9719136-debd-49fe-b1a6-5d6acdd67fb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:00.452493 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9719136-debd-49fe-b1a6-5d6acdd67fb3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:00.453148 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9719136-debd-49fe-b1a6-5d6acdd67fb3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:22:00.453452 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9719136-debd-49fe-b1a6-5d6acdd67fb3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:22:00.455038 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9719136-debd-49fe-b1a6-5d6acdd67fb3 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:22:00.475219 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1024/2083] 10.222.0.55 () {58 vars in 1153 bytes} [Fri Sep 23 15:22:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:22:00.479811 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9719136-debd-49fe-b1a6-5d6acdd67fb3 admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:22:00.490576 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9719136-debd-49fe-b1a6-5d6acdd67fb3 admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:22:00.495473 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e726b3-439e-4946-857a-2627f6b87171 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:22:00.497440 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e726b3-439e-4946-857a-2627f6b87171 admin admin] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:00.497440 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e726b3-439e-4946-857a-2627f6b87171 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:00.497440 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e726b3-439e-4946-857a-2627f6b87171 admin admin] PATH_INFO: `/v3/projects` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:00.498541 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e726b3-439e-4946-857a-2627f6b87171 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:22:00.500537 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e726b3-439e-4946-857a-2627f6b87171 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:22:00.513376 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac7fd19-30ac-4bc8-aeab-f1d6f8c53e25 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:22:00.517320 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1284fcb6-f8c6-4754-9f17-256796bd6218 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feca6278be904ef8b0dc580a67e477f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:22:00.517938 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1284fcb6-f8c6-4754-9f17-256796bd6218 service cinder] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:00.518226 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1284fcb6-f8c6-4754-9f17-256796bd6218 service cinder] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:00.518503 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1284fcb6-f8c6-4754-9f17-256796bd6218 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:00.528278 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1284fcb6-f8c6-4754-9f17-256796bd6218 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:22:00.529813 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1284fcb6-f8c6-4754-9f17-256796bd6218 service cinder] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:22:00.539643 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1060/2084] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:22:00 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 23 15:22:00.634607 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1025/2085] 10.222.0.55 () {60 vars in 1153 bytes} [Fri Sep 23 15:22:00 2022] POST /identity/v3/projects => generated 387 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:22:00.638620 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1284fcb6-f8c6-4754-9f17-256796bd6218 service cinder] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:22:00.649236 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1284fcb6-f8c6-4754-9f17-256796bd6218 service cinder] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:22:00.707814 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb72586-a0c1-4bec-aa9e-2a7454097590 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:22:00.708461 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb72586-a0c1-4bec-aa9e-2a7454097590 admin admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:00.708745 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb72586-a0c1-4bec-aa9e-2a7454097590 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:00.709019 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb72586-a0c1-4bec-aa9e-2a7454097590 admin admin] PATH_INFO: `/v3/users` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:00.709752 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb72586-a0c1-4bec-aa9e-2a7454097590 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:22:00.711642 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb72586-a0c1-4bec-aa9e-2a7454097590 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:22:00.852731 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:22:00.999497 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1061/2086] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Sep 23 15:22:00 2022] POST /identity/v3/users => generated 294 bytes in 362 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:22:01.003546 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e726b3-439e-4946-857a-2627f6b87171 admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:22:01.014792 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e726b3-439e-4946-857a-2627f6b87171 admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:22:01.016293 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb72586-a0c1-4bec-aa9e-2a7454097590 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:22:01.019376 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8e47b1-d0c0-4913-8d42-1688a16bcac5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:22:01.020000 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8e47b1-d0c0-4913-8d42-1688a16bcac5 admin admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:01.020288 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8e47b1-d0c0-4913-8d42-1688a16bcac5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:01.024225 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8e47b1-d0c0-4913-8d42-1688a16bcac5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:01.024225 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8e47b1-d0c0-4913-8d42-1688a16bcac5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:22:01.024225 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8e47b1-d0c0-4913-8d42-1688a16bcac5 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:22:01.033733 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1026/2087] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Sep 23 15:22:01 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:22:01.037679 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8e47b1-d0c0-4913-8d42-1688a16bcac5 admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:22:01.047212 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8e47b1-d0c0-4913-8d42-1688a16bcac5 admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:22:01.054056 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbbb3c6-fbde-4241-8680-0e279f5eac60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:22:01.054674 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbbb3c6-fbde-4241-8680-0e279f5eac60 admin admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:01.054949 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbbb3c6-fbde-4241-8680-0e279f5eac60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:01.055219 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbbb3c6-fbde-4241-8680-0e279f5eac60 admin admin] PATH_INFO: `/v3/roles` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:01.067377 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbbb3c6-fbde-4241-8680-0e279f5eac60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:22:01.067377 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbbb3c6-fbde-4241-8680-0e279f5eac60 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:22:01.068680 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1027/2088] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Sep 23 15:22:01 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 23 15:22:01.072987 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbbb3c6-fbde-4241-8680-0e279f5eac60 admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:22:01.084387 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbbb3c6-fbde-4241-8680-0e279f5eac60 admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:22:01.085430 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb72586-a0c1-4bec-aa9e-2a7454097590 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:22:01.089041 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deaebea2-fea1-458d-8358-2a881525a77d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:22:01.089207 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c887818b-8736-456a-a653-32280a43ca54 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feca6278be904ef8b0dc580a67e477f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:22:01.089696 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaebea2-fea1-458d-8358-2a881525a77d admin admin] REQUEST_METHOD: `PUT` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:01.089793 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c887818b-8736-456a-a653-32280a43ca54 service cinder] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:01.089977 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaebea2-fea1-458d-8358-2a881525a77d admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:01.090070 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c887818b-8736-456a-a653-32280a43ca54 service cinder] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:01.090252 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaebea2-fea1-458d-8358-2a881525a77d admin admin] PATH_INFO: `/v3/projects/e35e373ab18540df980ebba4d901e603/users/ee9e2bca1c2f43bab1f86501d9ae5caf/roles/af257be2dee54d6b942f58650196582c` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:01.090347 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c887818b-8736-456a-a653-32280a43ca54 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:01.092815 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-c887818b-8736-456a-a653-32280a43ca54 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:22:01.107543 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deaebea2-fea1-458d-8358-2a881525a77d admin admin] RBAC: Authorizing `identity:create_grant(project_id=e35e373ab18540df980ebba4d901e603, user_id=ee9e2bca1c2f43bab1f86501d9ae5caf, role_id=af257be2dee54d6b942f58650196582c)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:22:01.110236 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deaebea2-fea1-458d-8358-2a881525a77d admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:22:01.151677 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c887818b-8736-456a-a653-32280a43ca54 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:22:01.153105 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c887818b-8736-456a-a653-32280a43ca54 service cinder] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:22:01.162716 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1062/2089] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:22:01 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 23 15:22:01.337923 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1028/2090] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Sep 23 15:22:01 2022] PUT /identity/v3/projects/e35e373ab18540df980ebba4d901e603/users/ee9e2bca1c2f43bab1f86501d9ae5caf/roles/af257be2dee54d6b942f58650196582c => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:22:01.350755 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eaf3dfd-dec3-4515-b232-25f10ead1cc4 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:01.351100 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eaf3dfd-dec3-4515-b232-25f10ead1cc4 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:01.351391 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eaf3dfd-dec3-4515-b232-25f10ead1cc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:01.386802 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-7eaf3dfd-dec3-4515-b232-25f10ead1cc4 None None] MFA Rules not processed for user `ee9e2bca1c2f43bab1f86501d9ae5caf`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:22:01.388736 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-7eaf3dfd-dec3-4515-b232-25f10ead1cc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:22:01.457107 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1063/2091] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:22:01 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:22:01.462657 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deaebea2-fea1-458d-8358-2a881525a77d admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:22:01.537909 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deaebea2-fea1-458d-8358-2a881525a77d admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:22:01.543074 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198944ff-e1ef-40fc-bbd9-4da2b302f083 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:22:01.543716 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198944ff-e1ef-40fc-bbd9-4da2b302f083 admin admin] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:01.544235 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198944ff-e1ef-40fc-bbd9-4da2b302f083 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:01.544549 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198944ff-e1ef-40fc-bbd9-4da2b302f083 admin admin] PATH_INFO: `/v3/projects` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:01.545197 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198944ff-e1ef-40fc-bbd9-4da2b302f083 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:22:01.547123 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198944ff-e1ef-40fc-bbd9-4da2b302f083 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:22:01.664636 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1029/2092] 10.222.0.55 () {60 vars in 1153 bytes} [Fri Sep 23 15:22:01 2022] POST /identity/v3/projects => generated 387 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 23 15:22:01.666551 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eaf3dfd-dec3-4515-b232-25f10ead1cc4 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:22:01.677839 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eaf3dfd-dec3-4515-b232-25f10ead1cc4 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:22:01.709785 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=95,keystone:INSERT=1,keystone:DELETE=4 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:22:01.738470 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0aeb133-8bb4-4e91-92b3-06f638b90b05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:22:01.738902 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0aeb133-8bb4-4e91-92b3-06f638b90b05 admin admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:01.739198 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0aeb133-8bb4-4e91-92b3-06f638b90b05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:01.739478 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0aeb133-8bb4-4e91-92b3-06f638b90b05 admin admin] PATH_INFO: `/v3/users` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:01.740276 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0aeb133-8bb4-4e91-92b3-06f638b90b05 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:22:01.742188 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0aeb133-8bb4-4e91-92b3-06f638b90b05 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:22:01.928108 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1064/2093] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Sep 23 15:22:01 2022] POST /identity/v3/users => generated 300 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:22:01.929942 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198944ff-e1ef-40fc-bbd9-4da2b302f083 admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:22:01.941734 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198944ff-e1ef-40fc-bbd9-4da2b302f083 admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:22:01.946438 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b8e3a7-1bf2-45ee-9f6a-037ebda0a1fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:22:01.947049 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b8e3a7-1bf2-45ee-9f6a-037ebda0a1fa admin admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:01.947333 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b8e3a7-1bf2-45ee-9f6a-037ebda0a1fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:01.947607 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b8e3a7-1bf2-45ee-9f6a-037ebda0a1fa admin admin] PATH_INFO: `/v3/roles` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:01.948288 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b8e3a7-1bf2-45ee-9f6a-037ebda0a1fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:22:01.950015 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b8e3a7-1bf2-45ee-9f6a-037ebda0a1fa admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:22:01.962890 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1030/2094] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Sep 23 15:22:01 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:22:01.967678 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0aeb133-8bb4-4e91-92b3-06f638b90b05 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:22:01.977931 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0aeb133-8bb4-4e91-92b3-06f638b90b05 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:22:01.983128 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cd7d72-a4f9-4c99-849a-994399de5532 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:22:01.983784 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cd7d72-a4f9-4c99-849a-994399de5532 admin admin] REQUEST_METHOD: `PUT` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:01.984100 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cd7d72-a4f9-4c99-849a-994399de5532 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:01.984377 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cd7d72-a4f9-4c99-849a-994399de5532 admin admin] PATH_INFO: `/v3/projects/e04bc6f25a16463e9c5f9f8a25908d30/users/f552a03829d3455588e28d1fd0877adb/roles/8e2919e79df040e8926f21607818b58c` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:02.000090 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38cd7d72-a4f9-4c99-849a-994399de5532 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e04bc6f25a16463e9c5f9f8a25908d30, user_id=f552a03829d3455588e28d1fd0877adb, role_id=8e2919e79df040e8926f21607818b58c)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:22:02.002870 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38cd7d72-a4f9-4c99-849a-994399de5532 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:22:02.177347 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1065/2095] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Sep 23 15:22:01 2022] PUT /identity/v3/projects/e04bc6f25a16463e9c5f9f8a25908d30/users/f552a03829d3455588e28d1fd0877adb/roles/8e2919e79df040e8926f21607818b58c => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:22:02.181525 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b8e3a7-1bf2-45ee-9f6a-037ebda0a1fa admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:22:02.192617 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b8e3a7-1bf2-45ee-9f6a-037ebda0a1fa admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:22:02.258492 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a3d8ee-b5f0-4ce4-903b-eb202a830bc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:22:02.259394 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a3d8ee-b5f0-4ce4-903b-eb202a830bc4 admin admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:02.259677 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a3d8ee-b5f0-4ce4-903b-eb202a830bc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:02.260236 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a3d8ee-b5f0-4ce4-903b-eb202a830bc4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:02.261067 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a3d8ee-b5f0-4ce4-903b-eb202a830bc4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:22:02.264070 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a3d8ee-b5f0-4ce4-903b-eb202a830bc4 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:22:02.275817 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1031/2096] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Sep 23 15:22:02 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:22:02.288666 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cd7d72-a4f9-4c99-849a-994399de5532 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:22:02.304412 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cd7d72-a4f9-4c99-849a-994399de5532 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:22:02.310382 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43556781-ba9b-424b-8558-691455a48c8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:22:02.311154 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43556781-ba9b-424b-8558-691455a48c8d admin admin] REQUEST_METHOD: `PUT` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:02.311626 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43556781-ba9b-424b-8558-691455a48c8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:02.312120 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43556781-ba9b-424b-8558-691455a48c8d admin admin] PATH_INFO: `/v3/projects/e04bc6f25a16463e9c5f9f8a25908d30/users/f552a03829d3455588e28d1fd0877adb/roles/8e2919e79df040e8926f21607818b58c` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:02.315078 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43556781-ba9b-424b-8558-691455a48c8d admin admin] RBAC: Authorizing `identity:create_grant(project_id=e04bc6f25a16463e9c5f9f8a25908d30, user_id=f552a03829d3455588e28d1fd0877adb, role_id=8e2919e79df040e8926f21607818b58c)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:22:02.318532 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43556781-ba9b-424b-8558-691455a48c8d admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:22:02.333314 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1066/2097] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Sep 23 15:22:02 2022] PUT /identity/v3/projects/e04bc6f25a16463e9c5f9f8a25908d30/users/f552a03829d3455588e28d1fd0877adb/roles/8e2919e79df040e8926f21607818b58c => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 15:22:02.344259 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96527140-5334-4f07-b301-742b43db9045 None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:02.344840 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96527140-5334-4f07-b301-742b43db9045 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:02.345115 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96527140-5334-4f07-b301-742b43db9045 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:02.378172 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-96527140-5334-4f07-b301-742b43db9045 None None] MFA Rules not processed for user `f552a03829d3455588e28d1fd0877adb`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:22:02.379613 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-96527140-5334-4f07-b301-742b43db9045 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:22:02.527082 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1032/2098] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:22:02 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 15:22:02.543429 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43556781-ba9b-424b-8558-691455a48c8d admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:22:02.628690 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43556781-ba9b-424b-8558-691455a48c8d admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:22:02.632902 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a19cce-af92-41ec-81a0-9d98741110c9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feca6278be904ef8b0dc580a67e477f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:22:02.633816 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a19cce-af92-41ec-81a0-9d98741110c9 service cinder] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:02.634325 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a19cce-af92-41ec-81a0-9d98741110c9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:02.635070 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a19cce-af92-41ec-81a0-9d98741110c9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:02.649053 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a19cce-af92-41ec-81a0-9d98741110c9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:22:02.650506 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a19cce-af92-41ec-81a0-9d98741110c9 service cinder] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:22:02.673151 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1067/2099] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:22:02 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:22:02.737954 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96527140-5334-4f07-b301-742b43db9045 None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:22:02.802856 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96527140-5334-4f07-b301-742b43db9045 None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:22:02.807502 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2091c178-1108-4058-b223-1c0ac40307f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:22:02.808165 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2091c178-1108-4058-b223-1c0ac40307f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:02.808445 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2091c178-1108-4058-b223-1c0ac40307f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:02.808719 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2091c178-1108-4058-b223-1c0ac40307f3 admin admin] PATH_INFO: `/v3/users/ee9e2bca1c2f43bab1f86501d9ae5caf` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:02.810464 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2091c178-1108-4058-b223-1c0ac40307f3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ee9e2bca1c2f43bab1f86501d9ae5caf)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:22:02.812445 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2091c178-1108-4058-b223-1c0ac40307f3 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:22:03.061696 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:22:03.109713 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-2091c178-1108-4058-b223-1c0ac40307f3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ee9e2bca1c2f43bab1f86501d9ae5caf'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:22:03.122446 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-2091c178-1108-4058-b223-1c0ac40307f3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ee9e2bca1c2f43bab1f86501d9ae5caf'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:22:03.130301 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-2091c178-1108-4058-b223-1c0ac40307f3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ee9e2bca1c2f43bab1f86501d9ae5caf'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:22:03.278219 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-2091c178-1108-4058-b223-1c0ac40307f3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ee9e2bca1c2f43bab1f86501d9ae5caf'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:22:03.279295 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-2091c178-1108-4058-b223-1c0ac40307f3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ee9e2bca1c2f43bab1f86501d9ae5caf'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:22:03.286128 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1033/2100] 10.222.0.55 () {58 vars in 1224 bytes} [Fri Sep 23 15:22:02 2022] DELETE /identity/v3/users/ee9e2bca1c2f43bab1f86501d9ae5caf => generated 0 bytes in 549 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:22:03.291788 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ab8be5-512b-4290-940a-6ee24e4a2689 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:03.292407 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ab8be5-512b-4290-940a-6ee24e4a2689 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:03.292407 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ab8be5-512b-4290-940a-6ee24e4a2689 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:03.317140 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-19ab8be5-512b-4290-940a-6ee24e4a2689 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:22:03.318648 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-19ab8be5-512b-4290-940a-6ee24e4a2689 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:22:03.381417 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1068/2101] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:22:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:22:03.392473 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2091c178-1108-4058-b223-1c0ac40307f3 admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:22:03.394016 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-2091c178-1108-4058-b223-1c0ac40307f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:22:03.409986 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:22:03.453936 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2091c178-1108-4058-b223-1c0ac40307f3 admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:22:03.458959 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4500626-b1fc-43d3-be9f-2116a5367427 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd259178570864ffeb936ee10dfec6b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:22:03.459621 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4500626-b1fc-43d3-be9f-2116a5367427 service neutron] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:03.459905 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4500626-b1fc-43d3-be9f-2116a5367427 service neutron] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:03.460212 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4500626-b1fc-43d3-be9f-2116a5367427 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:03.478875 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4500626-b1fc-43d3-be9f-2116a5367427 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:22:03.480668 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4500626-b1fc-43d3-be9f-2116a5367427 service neutron] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:22:03.493254 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1034/2102] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:22:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:22:03.541999 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ab8be5-512b-4290-940a-6ee24e4a2689 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:22:03.543585 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-19ab8be5-512b-4290-940a-6ee24e4a2689 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:22:03.560658 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ab8be5-512b-4290-940a-6ee24e4a2689 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:22:03.570993 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae818c20-cbd0-4abc-97eb-5e3b18e7730a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:22:03.572064 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae818c20-cbd0-4abc-97eb-5e3b18e7730a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:03.572539 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae818c20-cbd0-4abc-97eb-5e3b18e7730a admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:03.572999 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae818c20-cbd0-4abc-97eb-5e3b18e7730a admin admin] PATH_INFO: `/v3/users/a0a92f89ac734b84b52ccf19e7ab9139` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:03.574447 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae818c20-cbd0-4abc-97eb-5e3b18e7730a admin admin] RBAC: Authorizing `identity:delete_user(user_id=a0a92f89ac734b84b52ccf19e7ab9139)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:22:03.576964 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae818c20-cbd0-4abc-97eb-5e3b18e7730a admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:22:03.708302 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4500626-b1fc-43d3-be9f-2116a5367427 service neutron] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:22:03.713178 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-b4500626-b1fc-43d3-be9f-2116a5367427 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:22:03.736384 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4500626-b1fc-43d3-be9f-2116a5367427 service neutron] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:22:03.756577 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2e3fd2-fa13-4a14-bbc8-950918b28788 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:22:03.757954 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2e3fd2-fa13-4a14-bbc8-950918b28788 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:03.759588 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2e3fd2-fa13-4a14-bbc8-950918b28788 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:03.762170 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2e3fd2-fa13-4a14-bbc8-950918b28788 admin admin] PATH_INFO: `/v3/users/ab6a43229be044669d705bc72edc96f4` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:03.767396 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2e3fd2-fa13-4a14-bbc8-950918b28788 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ab6a43229be044669d705bc72edc96f4)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:22:03.770500 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2e3fd2-fa13-4a14-bbc8-950918b28788 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:22:04.251020 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-ae818c20-cbd0-4abc-97eb-5e3b18e7730a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a0a92f89ac734b84b52ccf19e7ab9139'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:22:04.267121 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-ae818c20-cbd0-4abc-97eb-5e3b18e7730a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a0a92f89ac734b84b52ccf19e7ab9139'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:22:04.276880 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-ae818c20-cbd0-4abc-97eb-5e3b18e7730a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a0a92f89ac734b84b52ccf19e7ab9139'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:22:04.455513 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-ae818c20-cbd0-4abc-97eb-5e3b18e7730a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a0a92f89ac734b84b52ccf19e7ab9139'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:22:04.456610 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-ae818c20-cbd0-4abc-97eb-5e3b18e7730a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a0a92f89ac734b84b52ccf19e7ab9139'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:22:04.463196 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1069/2103] 10.222.0.55 () {58 vars in 1224 bytes} [Fri Sep 23 15:22:03 2022] DELETE /identity/v3/users/a0a92f89ac734b84b52ccf19e7ab9139 => generated 0 bytes in 922 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:22:04.464122 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae818c20-cbd0-4abc-97eb-5e3b18e7730a admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:22:04.465430 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-ae818c20-cbd0-4abc-97eb-5e3b18e7730a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:22:04.472459 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-2f2e3fd2-fa13-4a14-bbc8-950918b28788 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ab6a43229be044669d705bc72edc96f4'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:22:04.491368 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-2f2e3fd2-fa13-4a14-bbc8-950918b28788 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ab6a43229be044669d705bc72edc96f4'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:22:04.499285 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-2f2e3fd2-fa13-4a14-bbc8-950918b28788 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ab6a43229be044669d705bc72edc96f4'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:22:04.536125 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae818c20-cbd0-4abc-97eb-5e3b18e7730a admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:22:04.540884 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35742009-9a0e-4be0-8da2-c0d73988eb77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:22:04.541514 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35742009-9a0e-4be0-8da2-c0d73988eb77 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:04.541793 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35742009-9a0e-4be0-8da2-c0d73988eb77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:04.542068 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35742009-9a0e-4be0-8da2-c0d73988eb77 admin admin] PATH_INFO: `/v3/projects/e35e373ab18540df980ebba4d901e603` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:04.543189 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35742009-9a0e-4be0-8da2-c0d73988eb77 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e35e373ab18540df980ebba4d901e603)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:22:04.545115 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35742009-9a0e-4be0-8da2-c0d73988eb77 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:22:04.568691 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:22:04.602290 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-2f2e3fd2-fa13-4a14-bbc8-950918b28788 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ab6a43229be044669d705bc72edc96f4'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:22:04.602924 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-2f2e3fd2-fa13-4a14-bbc8-950918b28788 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ab6a43229be044669d705bc72edc96f4'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:22:04.610756 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1035/2104] 10.222.0.55 () {58 vars in 1224 bytes} [Fri Sep 23 15:22:03 2022] DELETE /identity/v3/users/ab6a43229be044669d705bc72edc96f4 => generated 0 bytes in 903 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:22:04.612114 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235ebfea-423f-4af0-8bd0-9395467df169 None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:04.612397 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235ebfea-423f-4af0-8bd0-9395467df169 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:04.612670 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235ebfea-423f-4af0-8bd0-9395467df169 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:04.633464 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-235ebfea-423f-4af0-8bd0-9395467df169 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:22:04.634979 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-235ebfea-423f-4af0-8bd0-9395467df169 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:22:04.700661 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1036/2105] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:22:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:22:04.702192 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c7c88e-40fb-4513-9ebf-df12a5a71e67 None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:04.702543 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c7c88e-40fb-4513-9ebf-df12a5a71e67 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:04.702820 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c7c88e-40fb-4513-9ebf-df12a5a71e67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:04.723286 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-d5c7c88e-40fb-4513-9ebf-df12a5a71e67 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:22:04.724339 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-35742009-9a0e-4be0-8da2-c0d73988eb77 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e35e373ab18540df980ebba4d901e603', 'request_id': 'req-35742009-9a0e-4be0-8da2-c0d73988eb77'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:22:04.724561 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-d5c7c88e-40fb-4513-9ebf-df12a5a71e67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:22:04.730514 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-35742009-9a0e-4be0-8da2-c0d73988eb77 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e35e373ab18540df980ebba4d901e603', 'request_id': 'req-35742009-9a0e-4be0-8da2-c0d73988eb77'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:22:04.738447 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-35742009-9a0e-4be0-8da2-c0d73988eb77 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e35e373ab18540df980ebba4d901e603', 'request_id': 'req-35742009-9a0e-4be0-8da2-c0d73988eb77'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:22:04.794677 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1037/2106] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:22:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:22:04.796553 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c7c88e-40fb-4513-9ebf-df12a5a71e67 None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:22:04.797940 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-d5c7c88e-40fb-4513-9ebf-df12a5a71e67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:22:04.875763 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c7c88e-40fb-4513-9ebf-df12a5a71e67 None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:22:04.879700 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947601bd-a84a-4ab4-9678-975803e67901 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd259178570864ffeb936ee10dfec6b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:22:04.880441 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947601bd-a84a-4ab4-9678-975803e67901 service neutron] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:04.881175 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947601bd-a84a-4ab4-9678-975803e67901 service neutron] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:04.881175 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947601bd-a84a-4ab4-9678-975803e67901 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:04.890925 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947601bd-a84a-4ab4-9678-975803e67901 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:22:04.892408 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947601bd-a84a-4ab4-9678-975803e67901 service neutron] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:22:04.901386 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-35742009-9a0e-4be0-8da2-c0d73988eb77 admin admin] The token cache is being invalidate because project e35e373ab18540df980ebba4d901e603 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81994) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 15:22:04.908672 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-35742009-9a0e-4be0-8da2-c0d73988eb77 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:22:04.908672 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-35742009-9a0e-4be0-8da2-c0d73988eb77 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:22:04.908874 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1038/2107] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:22:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:22:04.908874 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947601bd-a84a-4ab4-9678-975803e67901 service neutron] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:22:04.909047 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1070/2108] 10.222.0.55 () {58 vars in 1233 bytes} [Fri Sep 23 15:22:04 2022] DELETE /identity/v3/projects/e35e373ab18540df980ebba4d901e603 => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:22:04.909047 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35742009-9a0e-4be0-8da2-c0d73988eb77 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:22:04.909795 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-35742009-9a0e-4be0-8da2-c0d73988eb77 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:22:04.913732 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947601bd-a84a-4ab4-9678-975803e67901 service neutron] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:22:04.918243 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d378264b-a22a-4ccc-84bd-5f43282a0e6a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd259178570864ffeb936ee10dfec6b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:22:04.919046 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d378264b-a22a-4ccc-84bd-5f43282a0e6a service neutron] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:04.919472 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d378264b-a22a-4ccc-84bd-5f43282a0e6a service neutron] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:04.919815 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d378264b-a22a-4ccc-84bd-5f43282a0e6a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:04.922793 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-d378264b-a22a-4ccc-84bd-5f43282a0e6a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:22:04.926436 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35742009-9a0e-4be0-8da2-c0d73988eb77 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:22:04.931841 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b823d2a-b710-4801-9007-c46d69de8c09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:22:04.932728 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b823d2a-b710-4801-9007-c46d69de8c09 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:04.933194 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b823d2a-b710-4801-9007-c46d69de8c09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:04.933630 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b823d2a-b710-4801-9007-c46d69de8c09 admin admin] PATH_INFO: `/v3/users/f552a03829d3455588e28d1fd0877adb` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:04.935101 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b823d2a-b710-4801-9007-c46d69de8c09 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f552a03829d3455588e28d1fd0877adb)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:22:04.938115 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b823d2a-b710-4801-9007-c46d69de8c09 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:22:04.939821 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d378264b-a22a-4ccc-84bd-5f43282a0e6a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:22:04.941958 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d378264b-a22a-4ccc-84bd-5f43282a0e6a service neutron] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:22:04.960481 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1039/2109] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:22:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:22:05.268025 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-2b823d2a-b710-4801-9007-c46d69de8c09 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f552a03829d3455588e28d1fd0877adb'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:22:05.313985 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-2b823d2a-b710-4801-9007-c46d69de8c09 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f552a03829d3455588e28d1fd0877adb'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:22:05.319484 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-2b823d2a-b710-4801-9007-c46d69de8c09 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f552a03829d3455588e28d1fd0877adb'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:22:05.423325 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-2b823d2a-b710-4801-9007-c46d69de8c09 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f552a03829d3455588e28d1fd0877adb'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:22:05.427413 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-2b823d2a-b710-4801-9007-c46d69de8c09 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f552a03829d3455588e28d1fd0877adb'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:22:05.439362 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1071/2110] 10.222.0.55 () {58 vars in 1224 bytes} [Fri Sep 23 15:22:04 2022] DELETE /identity/v3/users/f552a03829d3455588e28d1fd0877adb => generated 0 bytes in 529 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 15:22:05.583080 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d378264b-a22a-4ccc-84bd-5f43282a0e6a service neutron] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:22:05.583080 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-d378264b-a22a-4ccc-84bd-5f43282a0e6a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:22:05.602893 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d378264b-a22a-4ccc-84bd-5f43282a0e6a service neutron] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:22:05.622907 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b823d2a-b710-4801-9007-c46d69de8c09 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:22:05.629425 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-2b823d2a-b710-4801-9007-c46d69de8c09 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:22:05.715536 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:22:05.734744 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0968b05-9584-4aef-865f-1596809ae13e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:22:05.735415 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0968b05-9584-4aef-865f-1596809ae13e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:05.735714 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0968b05-9584-4aef-865f-1596809ae13e admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:05.735971 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0968b05-9584-4aef-865f-1596809ae13e admin admin] PATH_INFO: `/v3/projects/078e0d3918e749e5aa9c00b1f3ce2114` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:05.737780 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0968b05-9584-4aef-865f-1596809ae13e admin admin] RBAC: Authorizing `identity:delete_project(project_id=078e0d3918e749e5aa9c00b1f3ce2114)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:22:05.739921 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0968b05-9584-4aef-865f-1596809ae13e admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:22:05.743600 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b823d2a-b710-4801-9007-c46d69de8c09 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:22:05.749344 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5f3f9f-1689-4c95-aa17-d2d51298a4e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:22:05.755150 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5f3f9f-1689-4c95-aa17-d2d51298a4e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:05.755150 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5f3f9f-1689-4c95-aa17-d2d51298a4e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:05.755150 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5f3f9f-1689-4c95-aa17-d2d51298a4e7 admin admin] PATH_INFO: `/v3/projects/f9ad8d24654c45b79bcb0bc4576476e8` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:05.755150 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5f3f9f-1689-4c95-aa17-d2d51298a4e7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f9ad8d24654c45b79bcb0bc4576476e8)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:22:05.755150 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5f3f9f-1689-4c95-aa17-d2d51298a4e7 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:22:05.874882 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-b0968b05-9584-4aef-865f-1596809ae13e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '078e0d3918e749e5aa9c00b1f3ce2114', 'request_id': 'req-b0968b05-9584-4aef-865f-1596809ae13e'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:22:05.881154 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-b0968b05-9584-4aef-865f-1596809ae13e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '078e0d3918e749e5aa9c00b1f3ce2114', 'request_id': 'req-b0968b05-9584-4aef-865f-1596809ae13e'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:22:05.942120 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-0c5f3f9f-1689-4c95-aa17-d2d51298a4e7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f9ad8d24654c45b79bcb0bc4576476e8', 'request_id': 'req-0c5f3f9f-1689-4c95-aa17-d2d51298a4e7'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:22:05.948574 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-0c5f3f9f-1689-4c95-aa17-d2d51298a4e7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f9ad8d24654c45b79bcb0bc4576476e8', 'request_id': 'req-0c5f3f9f-1689-4c95-aa17-d2d51298a4e7'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:22:05.955760 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:22:05.955760 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-b0968b05-9584-4aef-865f-1596809ae13e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '078e0d3918e749e5aa9c00b1f3ce2114', 'request_id': 'req-b0968b05-9584-4aef-865f-1596809ae13e'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:22:05.962226 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-0c5f3f9f-1689-4c95-aa17-d2d51298a4e7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f9ad8d24654c45b79bcb0bc4576476e8', 'request_id': 'req-0c5f3f9f-1689-4c95-aa17-d2d51298a4e7'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:22:05.971612 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-b0968b05-9584-4aef-865f-1596809ae13e admin admin] The token cache is being invalidate because project 078e0d3918e749e5aa9c00b1f3ce2114 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81993) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 15:22:05.972710 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-b0968b05-9584-4aef-865f-1596809ae13e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:22:05.973458 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-b0968b05-9584-4aef-865f-1596809ae13e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:22:05.975187 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1040/2111] 10.222.0.55 () {58 vars in 1233 bytes} [Fri Sep 23 15:22:05 2022] DELETE /identity/v3/projects/078e0d3918e749e5aa9c00b1f3ce2114 => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:22:05.980457 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0968b05-9584-4aef-865f-1596809ae13e admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:22:05.984927 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-b0968b05-9584-4aef-865f-1596809ae13e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:22:06.054945 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0968b05-9584-4aef-865f-1596809ae13e admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:22:06.117039 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-0c5f3f9f-1689-4c95-aa17-d2d51298a4e7 admin admin] The token cache is being invalidate because project f9ad8d24654c45b79bcb0bc4576476e8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81994) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 15:22:06.117039 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-0c5f3f9f-1689-4c95-aa17-d2d51298a4e7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:22:06.117039 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-0c5f3f9f-1689-4c95-aa17-d2d51298a4e7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:22:06.117039 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1072/2112] 10.222.0.55 () {58 vars in 1233 bytes} [Fri Sep 23 15:22:05 2022] DELETE /identity/v3/projects/f9ad8d24654c45b79bcb0bc4576476e8 => generated 0 bytes in 487 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:22:06.117039 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5f3f9f-1689-4c95-aa17-d2d51298a4e7 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:22:06.117471 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-0c5f3f9f-1689-4c95-aa17-d2d51298a4e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:22:06.121252 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9106fa1-cedc-43c7-8ecc-57224f566ac5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:22:06.121972 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9106fa1-cedc-43c7-8ecc-57224f566ac5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:06.122322 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9106fa1-cedc-43c7-8ecc-57224f566ac5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:06.122662 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9106fa1-cedc-43c7-8ecc-57224f566ac5 admin admin] PATH_INFO: `/v3/users/dac4337d429a4c41a91629e8d5ae1fed` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:06.124007 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9106fa1-cedc-43c7-8ecc-57224f566ac5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dac4337d429a4c41a91629e8d5ae1fed)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:22:06.126007 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9106fa1-cedc-43c7-8ecc-57224f566ac5 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:22:06.192787 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5f3f9f-1689-4c95-aa17-d2d51298a4e7 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:22:06.198436 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a057e4f-3831-49d0-a094-4bc24fd1b022 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:22:06.199080 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a057e4f-3831-49d0-a094-4bc24fd1b022 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:06.199359 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a057e4f-3831-49d0-a094-4bc24fd1b022 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:06.199630 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a057e4f-3831-49d0-a094-4bc24fd1b022 admin admin] PATH_INFO: `/v3/projects/e04bc6f25a16463e9c5f9f8a25908d30` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:06.200837 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a057e4f-3831-49d0-a094-4bc24fd1b022 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e04bc6f25a16463e9c5f9f8a25908d30)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:22:06.225866 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a057e4f-3831-49d0-a094-4bc24fd1b022 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:22:06.520426 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-d9106fa1-cedc-43c7-8ecc-57224f566ac5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dac4337d429a4c41a91629e8d5ae1fed'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:22:06.522091 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-8a057e4f-3831-49d0-a094-4bc24fd1b022 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e04bc6f25a16463e9c5f9f8a25908d30', 'request_id': 'req-8a057e4f-3831-49d0-a094-4bc24fd1b022'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:22:06.528563 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-8a057e4f-3831-49d0-a094-4bc24fd1b022 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e04bc6f25a16463e9c5f9f8a25908d30', 'request_id': 'req-8a057e4f-3831-49d0-a094-4bc24fd1b022'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:22:06.533975 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-d9106fa1-cedc-43c7-8ecc-57224f566ac5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dac4337d429a4c41a91629e8d5ae1fed'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:22:06.534876 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-8a057e4f-3831-49d0-a094-4bc24fd1b022 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e04bc6f25a16463e9c5f9f8a25908d30', 'request_id': 'req-8a057e4f-3831-49d0-a094-4bc24fd1b022'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:22:06.541008 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-d9106fa1-cedc-43c7-8ecc-57224f566ac5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dac4337d429a4c41a91629e8d5ae1fed'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:22:06.622579 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-8a057e4f-3831-49d0-a094-4bc24fd1b022 admin admin] The token cache is being invalidate because project e04bc6f25a16463e9c5f9f8a25908d30 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81994) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 15:22:06.622579 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-8a057e4f-3831-49d0-a094-4bc24fd1b022 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:22:06.622579 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-8a057e4f-3831-49d0-a094-4bc24fd1b022 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:22:06.622579 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1073/2113] 10.222.0.55 () {58 vars in 1233 bytes} [Fri Sep 23 15:22:06 2022] DELETE /identity/v3/projects/e04bc6f25a16463e9c5f9f8a25908d30 => generated 0 bytes in 509 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:22:06.622579 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d03ddee-eddc-4342-94ba-4dd4eaf04685 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:06.623452 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d03ddee-eddc-4342-94ba-4dd4eaf04685 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:06.623452 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d03ddee-eddc-4342-94ba-4dd4eaf04685 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:06.643851 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-6d03ddee-eddc-4342-94ba-4dd4eaf04685 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:22:06.645196 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-6d03ddee-eddc-4342-94ba-4dd4eaf04685 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:22:06.683744 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-d9106fa1-cedc-43c7-8ecc-57224f566ac5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dac4337d429a4c41a91629e8d5ae1fed'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:22:06.684402 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-d9106fa1-cedc-43c7-8ecc-57224f566ac5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dac4337d429a4c41a91629e8d5ae1fed'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:22:06.691838 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1041/2114] 10.222.0.55 () {58 vars in 1224 bytes} [Fri Sep 23 15:22:05 2022] DELETE /identity/v3/users/dac4337d429a4c41a91629e8d5ae1fed => generated 0 bytes in 712 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:22:06.696691 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25454428-be3c-4e89-9469-795efccca94f None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:06.696691 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25454428-be3c-4e89-9469-795efccca94f None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:06.696691 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25454428-be3c-4e89-9469-795efccca94f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:06.719201 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1074/2115] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:22:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:22:06.728564 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc48a76-df1e-49a2-8392-3ec1f208e74f None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:06.729665 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc48a76-df1e-49a2-8392-3ec1f208e74f None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:06.730231 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc48a76-df1e-49a2-8392-3ec1f208e74f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:06.732161 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-25454428-be3c-4e89-9469-795efccca94f None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:22:06.742489 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-25454428-be3c-4e89-9469-795efccca94f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:22:06.760230 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-fdc48a76-df1e-49a2-8392-3ec1f208e74f None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:22:06.761871 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-fdc48a76-df1e-49a2-8392-3ec1f208e74f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:22:06.822240 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1042/2116] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:22:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:22:06.830457 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17206343-5e69-4501-b164-9d61cb220615 None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:06.830746 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17206343-5e69-4501-b164-9d61cb220615 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:06.831162 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17206343-5e69-4501-b164-9d61cb220615 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:06.858423 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1075/2117] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:22:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 15:22:06.862419 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-17206343-5e69-4501-b164-9d61cb220615 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:22:06.863683 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-17206343-5e69-4501-b164-9d61cb220615 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:22:06.864590 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc48a76-df1e-49a2-8392-3ec1f208e74f None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:22:06.872973 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1043/2118] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:22:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 15:22:06.874793 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc48a76-df1e-49a2-8392-3ec1f208e74f None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:22:06.877681 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17206343-5e69-4501-b164-9d61cb220615 None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:22:06.880190 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4afec6c-c20a-4858-9466-aa65aae4d561 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:22:06.881255 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4afec6c-c20a-4858-9466-aa65aae4d561 admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:06.881823 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4afec6c-c20a-4858-9466-aa65aae4d561 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:06.882360 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4afec6c-c20a-4858-9466-aa65aae4d561 admin admin] PATH_INFO: `/v3/domains` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:06.883168 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4afec6c-c20a-4858-9466-aa65aae4d561 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:22:06.883514 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4afec6c-c20a-4858-9466-aa65aae4d561 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:22:06.885727 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4afec6c-c20a-4858-9466-aa65aae4d561 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:22:06.888501 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17206343-5e69-4501-b164-9d61cb220615 None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:22:06.893928 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fe2994-d96f-478d-8663-149429ddbdfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:22:06.896562 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fe2994-d96f-478d-8663-149429ddbdfc admin admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:06.896562 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fe2994-d96f-478d-8663-149429ddbdfc admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:06.896562 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fe2994-d96f-478d-8663-149429ddbdfc admin admin] PATH_INFO: `/v3/domains` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:06.896562 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fe2994-d96f-478d-8663-149429ddbdfc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:22:06.896562 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fe2994-d96f-478d-8663-149429ddbdfc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:22:06.897926 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fe2994-d96f-478d-8663-149429ddbdfc admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:22:06.901163 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1076/2119] 10.222.0.55 () {58 vars in 1153 bytes} [Fri Sep 23 15:22:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 15:22:06.905592 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4afec6c-c20a-4858-9466-aa65aae4d561 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:22:06.912007 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1044/2120] 10.222.0.55 () {58 vars in 1153 bytes} [Fri Sep 23 15:22:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:22:06.914878 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fe2994-d96f-478d-8663-149429ddbdfc admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:22:06.915872 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4afec6c-c20a-4858-9466-aa65aae4d561 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:22:06.923161 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f97732-31e5-4b0f-91ea-68e7e520a921 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:22:06.923858 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f97732-31e5-4b0f-91ea-68e7e520a921 admin admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:06.924203 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f97732-31e5-4b0f-91ea-68e7e520a921 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:06.924478 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f97732-31e5-4b0f-91ea-68e7e520a921 admin admin] PATH_INFO: `/v3/projects` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:06.925431 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fe2994-d96f-478d-8663-149429ddbdfc admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:22:06.925967 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f97732-31e5-4b0f-91ea-68e7e520a921 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:22:06.927936 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f97732-31e5-4b0f-91ea-68e7e520a921 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:22:06.930369 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c385d7a-53c3-489c-a8a7-61b9a1679651 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:22:06.930962 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c385d7a-53c3-489c-a8a7-61b9a1679651 admin admin] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:06.931272 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c385d7a-53c3-489c-a8a7-61b9a1679651 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:06.931544 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c385d7a-53c3-489c-a8a7-61b9a1679651 admin admin] PATH_INFO: `/v3/projects` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:06.932215 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c385d7a-53c3-489c-a8a7-61b9a1679651 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:22:06.934212 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c385d7a-53c3-489c-a8a7-61b9a1679651 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:22:07.143863 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1045/2121] 10.222.0.55 () {60 vars in 1153 bytes} [Fri Sep 23 15:22:06 2022] POST /identity/v3/projects => generated 401 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 23 15:22:07.149475 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c385d7a-53c3-489c-a8a7-61b9a1679651 admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:22:07.190739 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1077/2122] 10.222.0.55 () {60 vars in 1153 bytes} [Fri Sep 23 15:22:06 2022] POST /identity/v3/projects => generated 373 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 23 15:22:07.195648 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f97732-31e5-4b0f-91ea-68e7e520a921 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:22:07.226598 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c385d7a-53c3-489c-a8a7-61b9a1679651 admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:22:07.301823 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75564fcc-7eb2-4d90-abad-5cc83838bdb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:22:07.303254 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75564fcc-7eb2-4d90-abad-5cc83838bdb1 admin admin] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:07.303473 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75564fcc-7eb2-4d90-abad-5cc83838bdb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:07.303751 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75564fcc-7eb2-4d90-abad-5cc83838bdb1 admin admin] PATH_INFO: `/v3/users` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:07.304502 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75564fcc-7eb2-4d90-abad-5cc83838bdb1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:22:07.306353 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f97732-31e5-4b0f-91ea-68e7e520a921 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:22:07.306469 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75564fcc-7eb2-4d90-abad-5cc83838bdb1 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:22:07.311203 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e78058-574b-42ca-aeed-7cb6c7b599a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:22:07.312416 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e78058-574b-42ca-aeed-7cb6c7b599a9 admin admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:07.312689 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e78058-574b-42ca-aeed-7cb6c7b599a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:07.313062 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e78058-574b-42ca-aeed-7cb6c7b599a9 admin admin] PATH_INFO: `/v3/users` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:07.313658 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e78058-574b-42ca-aeed-7cb6c7b599a9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:22:07.315675 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e78058-574b-42ca-aeed-7cb6c7b599a9 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:22:07.441722 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1046/2123] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Sep 23 15:22:07 2022] POST /identity/v3/users => generated 301 bytes in 290 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 23 15:22:07.441722 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75564fcc-7eb2-4d90-abad-5cc83838bdb1 admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:22:07.441722 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-75564fcc-7eb2-4d90-abad-5cc83838bdb1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:22:07.451401 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1078/2124] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Sep 23 15:22:07 2022] POST /identity/v3/users => generated 287 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:22:07.453201 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e78058-574b-42ca-aeed-7cb6c7b599a9 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:22:07.457985 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75564fcc-7eb2-4d90-abad-5cc83838bdb1 admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:22:07.465200 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e78058-574b-42ca-aeed-7cb6c7b599a9 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:22:07.465329 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775d1e1b-9509-4994-9b77-31c8018470f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:22:07.466408 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775d1e1b-9509-4994-9b77-31c8018470f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:07.466696 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775d1e1b-9509-4994-9b77-31c8018470f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:07.466977 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775d1e1b-9509-4994-9b77-31c8018470f1 admin admin] PATH_INFO: `/v3/projects/33fe28df3f0547da9b76abb87b55902d` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:07.469356 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775d1e1b-9509-4994-9b77-31c8018470f1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=33fe28df3f0547da9b76abb87b55902d)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:22:07.470287 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcfb81e-e52b-43cb-a946-07d46a28c150 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:22:07.470919 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcfb81e-e52b-43cb-a946-07d46a28c150 admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:07.471198 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcfb81e-e52b-43cb-a946-07d46a28c150 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:07.471339 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775d1e1b-9509-4994-9b77-31c8018470f1 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:22:07.471460 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcfb81e-e52b-43cb-a946-07d46a28c150 admin admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:07.473272 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcfb81e-e52b-43cb-a946-07d46a28c150 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:22:07.475087 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcfb81e-e52b-43cb-a946-07d46a28c150 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:22:07.487322 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1079/2125] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Sep 23 15:22:07 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:22:07.488333 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcfb81e-e52b-43cb-a946-07d46a28c150 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:22:07.499656 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcfb81e-e52b-43cb-a946-07d46a28c150 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:22:07.506525 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d86b90-1904-4b5b-a047-f031110a63fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:22:07.507283 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d86b90-1904-4b5b-a047-f031110a63fe admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:07.508195 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d86b90-1904-4b5b-a047-f031110a63fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:07.508561 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d86b90-1904-4b5b-a047-f031110a63fe admin admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:07.512404 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d86b90-1904-4b5b-a047-f031110a63fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:22:07.514767 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d86b90-1904-4b5b-a047-f031110a63fe admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:22:07.530804 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1080/2126] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Sep 23 15:22:07 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:22:07.532041 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d86b90-1904-4b5b-a047-f031110a63fe admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:22:07.543283 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d86b90-1904-4b5b-a047-f031110a63fe admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:22:07.549025 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f20331b-f97b-474d-bbf8-6b6114896a1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:22:07.549635 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f20331b-f97b-474d-bbf8-6b6114896a1d admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:07.549919 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f20331b-f97b-474d-bbf8-6b6114896a1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:07.550192 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f20331b-f97b-474d-bbf8-6b6114896a1d admin admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:07.551263 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f20331b-f97b-474d-bbf8-6b6114896a1d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:22:07.552932 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f20331b-f97b-474d-bbf8-6b6114896a1d admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:22:07.565242 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1081/2127] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Sep 23 15:22:07 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:22:07.566235 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f20331b-f97b-474d-bbf8-6b6114896a1d admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:22:07.578203 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f20331b-f97b-474d-bbf8-6b6114896a1d admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:22:07.583566 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b10a3f-0d87-4cd6-842b-67634dcb2566 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:22:07.584248 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b10a3f-0d87-4cd6-842b-67634dcb2566 admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:07.584579 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b10a3f-0d87-4cd6-842b-67634dcb2566 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:07.584832 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b10a3f-0d87-4cd6-842b-67634dcb2566 admin admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:07.585463 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b10a3f-0d87-4cd6-842b-67634dcb2566 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:22:07.587311 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b10a3f-0d87-4cd6-842b-67634dcb2566 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:22:07.599240 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1082/2128] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Sep 23 15:22:07 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:22:07.600144 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b10a3f-0d87-4cd6-842b-67634dcb2566 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:22:07.612748 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b10a3f-0d87-4cd6-842b-67634dcb2566 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:22:07.618834 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b8d411-4836-41b2-bf64-b319120e42f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:22:07.619530 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b8d411-4836-41b2-bf64-b319120e42f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:07.620633 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b8d411-4836-41b2-bf64-b319120e42f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:07.621016 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b8d411-4836-41b2-bf64-b319120e42f3 admin admin] PATH_INFO: `/v3/projects/4e8b30906b244e4fb7bcd5da9b8fb279/users/ae62db6a9d164a2a8a9602b8c544c0b0/roles/af257be2dee54d6b942f58650196582c` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:07.649857 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b8d411-4836-41b2-bf64-b319120e42f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e8b30906b244e4fb7bcd5da9b8fb279, user_id=ae62db6a9d164a2a8a9602b8c544c0b0, role_id=af257be2dee54d6b942f58650196582c)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:22:07.655248 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b8d411-4836-41b2-bf64-b319120e42f3 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:22:07.663120 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-775d1e1b-9509-4994-9b77-31c8018470f1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '33fe28df3f0547da9b76abb87b55902d', 'request_id': 'req-775d1e1b-9509-4994-9b77-31c8018470f1'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:22:07.669147 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-775d1e1b-9509-4994-9b77-31c8018470f1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '33fe28df3f0547da9b76abb87b55902d', 'request_id': 'req-775d1e1b-9509-4994-9b77-31c8018470f1'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:22:07.689873 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:22:07.780905 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1083/2129] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Sep 23 15:22:07 2022] PUT /identity/v3/projects/4e8b30906b244e4fb7bcd5da9b8fb279/users/ae62db6a9d164a2a8a9602b8c544c0b0/roles/af257be2dee54d6b942f58650196582c => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:22:07.781976 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b8d411-4836-41b2-bf64-b319120e42f3 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:22:07.818514 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-775d1e1b-9509-4994-9b77-31c8018470f1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '33fe28df3f0547da9b76abb87b55902d', 'request_id': 'req-775d1e1b-9509-4994-9b77-31c8018470f1'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:22:07.831431 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-775d1e1b-9509-4994-9b77-31c8018470f1 admin admin] The token cache is being invalidate because project 33fe28df3f0547da9b76abb87b55902d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81993) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 15:22:07.831718 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-775d1e1b-9509-4994-9b77-31c8018470f1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:22:07.832407 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-775d1e1b-9509-4994-9b77-31c8018470f1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:22:07.834422 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1047/2130] 10.222.0.55 () {58 vars in 1233 bytes} [Fri Sep 23 15:22:07 2022] DELETE /identity/v3/projects/33fe28df3f0547da9b76abb87b55902d => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:22:07.835336 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c267279-6cc8-4a20-af6d-979a8be48a1d None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:07.835790 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c267279-6cc8-4a20-af6d-979a8be48a1d None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:07.836120 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c267279-6cc8-4a20-af6d-979a8be48a1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:07.879510 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-3c267279-6cc8-4a20-af6d-979a8be48a1d None None] MFA Rules not processed for user `ae62db6a9d164a2a8a9602b8c544c0b0`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:22:07.880782 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-3c267279-6cc8-4a20-af6d-979a8be48a1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:22:07.883446 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b8d411-4836-41b2-bf64-b319120e42f3 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:22:07.889430 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7eb473e-38bf-45be-a01f-0b168d68855d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:22:07.890220 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7eb473e-38bf-45be-a01f-0b168d68855d admin admin] REQUEST_METHOD: `PUT` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:07.890989 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7eb473e-38bf-45be-a01f-0b168d68855d admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:07.891178 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7eb473e-38bf-45be-a01f-0b168d68855d admin admin] PATH_INFO: `/v3/projects/7c95b871ff2b4ed89195a9c0fa94e8b4/users/4bdaecd5dd5d461383878c4b96ef98be/roles/af257be2dee54d6b942f58650196582c` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:07.918352 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7eb473e-38bf-45be-a01f-0b168d68855d admin admin] RBAC: Authorizing `identity:create_grant(project_id=7c95b871ff2b4ed89195a9c0fa94e8b4, user_id=4bdaecd5dd5d461383878c4b96ef98be, role_id=af257be2dee54d6b942f58650196582c)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:22:07.921086 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7eb473e-38bf-45be-a01f-0b168d68855d admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:22:07.963071 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1048/2131] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:22:07 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:22:07.964386 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575c8217-a6c3-4fb8-99af-ff1927171ee5 None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:07.964686 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575c8217-a6c3-4fb8-99af-ff1927171ee5 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:07.964954 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575c8217-a6c3-4fb8-99af-ff1927171ee5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:07.985282 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-575c8217-a6c3-4fb8-99af-ff1927171ee5 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:22:07.989018 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-575c8217-a6c3-4fb8-99af-ff1927171ee5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:22:07.995199 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1049/2132] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:22:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:22:07.996029 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575c8217-a6c3-4fb8-99af-ff1927171ee5 None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:22:07.997276 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-575c8217-a6c3-4fb8-99af-ff1927171ee5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:22:08.010679 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575c8217-a6c3-4fb8-99af-ff1927171ee5 None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:22:08.016002 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7379b56f-48de-4c36-b908-72b955cb9994 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:22:08.017875 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7379b56f-48de-4c36-b908-72b955cb9994 admin admin] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:08.017996 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7379b56f-48de-4c36-b908-72b955cb9994 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:08.018270 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7379b56f-48de-4c36-b908-72b955cb9994 admin admin] PATH_INFO: `/v3/projects` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:08.018926 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7379b56f-48de-4c36-b908-72b955cb9994 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:22:08.021041 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7379b56f-48de-4c36-b908-72b955cb9994 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:22:08.125356 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:22:08.132069 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1084/2133] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Sep 23 15:22:07 2022] PUT /identity/v3/projects/7c95b871ff2b4ed89195a9c0fa94e8b4/users/4bdaecd5dd5d461383878c4b96ef98be/roles/af257be2dee54d6b942f58650196582c => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:22:08.133424 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4291ecaf-6a8c-4c2b-8873-da7b1f60eee9 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:08.133728 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4291ecaf-6a8c-4c2b-8873-da7b1f60eee9 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:08.136011 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4291ecaf-6a8c-4c2b-8873-da7b1f60eee9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:08.156795 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-4291ecaf-6a8c-4c2b-8873-da7b1f60eee9 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:22:08.158101 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-4291ecaf-6a8c-4c2b-8873-da7b1f60eee9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:22:08.164760 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1050/2134] 10.222.0.55 () {60 vars in 1153 bytes} [Fri Sep 23 15:22:07 2022] POST /identity/v3/projects => generated 401 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:22:08.169191 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1382f93-d10d-47af-aab1-f525917db260 None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:08.169470 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1382f93-d10d-47af-aab1-f525917db260 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:08.169752 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1382f93-d10d-47af-aab1-f525917db260 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:08.189589 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-c1382f93-d10d-47af-aab1-f525917db260 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:22:08.190852 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-c1382f93-d10d-47af-aab1-f525917db260 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:22:08.218892 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1085/2135] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:22:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:22:08.219532 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4291ecaf-6a8c-4c2b-8873-da7b1f60eee9 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:22:08.228839 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4291ecaf-6a8c-4c2b-8873-da7b1f60eee9 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:22:08.262080 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1051/2136] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:22:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:22:08.263244 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1382f93-d10d-47af-aab1-f525917db260 None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:22:08.276153 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1382f93-d10d-47af-aab1-f525917db260 None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:22:08.283811 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d794e5-1de3-43d6-bfe6-61d838a125ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:22:08.285779 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d794e5-1de3-43d6-bfe6-61d838a125ca admin admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:08.285779 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d794e5-1de3-43d6-bfe6-61d838a125ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:08.285779 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d794e5-1de3-43d6-bfe6-61d838a125ca admin admin] PATH_INFO: `/v3/domains` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:08.285779 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d794e5-1de3-43d6-bfe6-61d838a125ca admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:22:08.286141 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d794e5-1de3-43d6-bfe6-61d838a125ca admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:22:08.287680 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d794e5-1de3-43d6-bfe6-61d838a125ca admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:22:08.308163 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1052/2137] 10.222.0.55 () {58 vars in 1153 bytes} [Fri Sep 23 15:22:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:22:08.308163 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d794e5-1de3-43d6-bfe6-61d838a125ca admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:22:08.312083 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-76d794e5-1de3-43d6-bfe6-61d838a125ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:22:08.316963 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13be6d3f-6bf2-463d-b4c3-24cae2cb157c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:22:08.317615 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13be6d3f-6bf2-463d-b4c3-24cae2cb157c admin admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:08.318065 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13be6d3f-6bf2-463d-b4c3-24cae2cb157c admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:08.318503 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13be6d3f-6bf2-463d-b4c3-24cae2cb157c admin admin] PATH_INFO: `/v3/users` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:08.319237 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13be6d3f-6bf2-463d-b4c3-24cae2cb157c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:22:08.321300 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13be6d3f-6bf2-463d-b4c3-24cae2cb157c admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:22:08.390735 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d794e5-1de3-43d6-bfe6-61d838a125ca admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:22:08.395250 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09f5dc4-472b-4f8f-8aae-e79cf738e835 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd259178570864ffeb936ee10dfec6b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:22:08.396024 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09f5dc4-472b-4f8f-8aae-e79cf738e835 service neutron] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:08.396377 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09f5dc4-472b-4f8f-8aae-e79cf738e835 service neutron] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:08.397185 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09f5dc4-472b-4f8f-8aae-e79cf738e835 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:08.406668 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09f5dc4-472b-4f8f-8aae-e79cf738e835 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:22:08.408199 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09f5dc4-472b-4f8f-8aae-e79cf738e835 service neutron] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:22:08.418373 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1053/2138] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:22:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:22:08.419202 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09f5dc4-472b-4f8f-8aae-e79cf738e835 service neutron] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:22:08.432646 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09f5dc4-472b-4f8f-8aae-e79cf738e835 service neutron] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:22:08.438063 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4a2645-b271-44c9-862e-111754c7961f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:22:08.439249 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4a2645-b271-44c9-862e-111754c7961f admin admin] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:08.439530 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4a2645-b271-44c9-862e-111754c7961f admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:08.439807 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4a2645-b271-44c9-862e-111754c7961f admin admin] PATH_INFO: `/v3/projects` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:08.440584 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4a2645-b271-44c9-862e-111754c7961f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:22:08.442667 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4a2645-b271-44c9-862e-111754c7961f admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:22:08.518125 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1086/2139] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Sep 23 15:22:08 2022] POST /identity/v3/users => generated 307 bytes in 299 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:22:08.522009 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13be6d3f-6bf2-463d-b4c3-24cae2cb157c admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:22:08.533121 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13be6d3f-6bf2-463d-b4c3-24cae2cb157c admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:22:08.538375 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6219e28-abdc-45fa-aa7f-7537acd672c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:22:08.539094 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6219e28-abdc-45fa-aa7f-7537acd672c0 admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:08.539431 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6219e28-abdc-45fa-aa7f-7537acd672c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:08.539928 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6219e28-abdc-45fa-aa7f-7537acd672c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:08.540807 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6219e28-abdc-45fa-aa7f-7537acd672c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:22:08.542460 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6219e28-abdc-45fa-aa7f-7537acd672c0 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:22:08.555003 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1087/2140] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Sep 23 15:22:08 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 23 15:22:08.559144 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6219e28-abdc-45fa-aa7f-7537acd672c0 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:22:08.570358 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6219e28-abdc-45fa-aa7f-7537acd672c0 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:22:08.585886 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e650a14-5942-4ab7-9b64-3a4c546e9053 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:22:08.585886 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e650a14-5942-4ab7-9b64-3a4c546e9053 admin admin] REQUEST_METHOD: `PUT` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:08.585886 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e650a14-5942-4ab7-9b64-3a4c546e9053 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:08.585886 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e650a14-5942-4ab7-9b64-3a4c546e9053 admin admin] PATH_INFO: `/v3/projects/130c4dd2e5424c12b0906656c39bc9b6/users/76ed9575994949f58e54209f019a475e/roles/8e2919e79df040e8926f21607818b58c` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:08.592483 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e650a14-5942-4ab7-9b64-3a4c546e9053 admin admin] RBAC: Authorizing `identity:create_grant(project_id=130c4dd2e5424c12b0906656c39bc9b6, user_id=76ed9575994949f58e54209f019a475e, role_id=8e2919e79df040e8926f21607818b58c)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:22:08.595413 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e650a14-5942-4ab7-9b64-3a4c546e9053 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:22:08.688328 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1054/2141] 10.222.0.55 () {60 vars in 1153 bytes} [Fri Sep 23 15:22:08 2022] POST /identity/v3/projects => generated 369 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:22:08.690428 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4a2645-b271-44c9-862e-111754c7961f admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:22:08.758149 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1088/2142] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Sep 23 15:22:08 2022] PUT /identity/v3/projects/130c4dd2e5424c12b0906656c39bc9b6/users/76ed9575994949f58e54209f019a475e/roles/8e2919e79df040e8926f21607818b58c => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:22:08.760952 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e650a14-5942-4ab7-9b64-3a4c546e9053 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:22:08.763276 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4a2645-b271-44c9-862e-111754c7961f admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:22:08.877440 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafa4c7d-e28b-4ef0-8af9-23a80441ad02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:22:08.881009 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafa4c7d-e28b-4ef0-8af9-23a80441ad02 admin admin] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:08.881009 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafa4c7d-e28b-4ef0-8af9-23a80441ad02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:08.881009 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafa4c7d-e28b-4ef0-8af9-23a80441ad02 admin admin] PATH_INFO: `/v3/users` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:08.881009 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafa4c7d-e28b-4ef0-8af9-23a80441ad02 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:22:08.881423 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e650a14-5942-4ab7-9b64-3a4c546e9053 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:22:08.882179 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafa4c7d-e28b-4ef0-8af9-23a80441ad02 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:22:08.885600 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4156ee8-793a-4d3c-92ab-f0f1d7aa5ad7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:22:08.886226 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4156ee8-793a-4d3c-92ab-f0f1d7aa5ad7 admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:08.886591 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4156ee8-793a-4d3c-92ab-f0f1d7aa5ad7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:08.887160 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4156ee8-793a-4d3c-92ab-f0f1d7aa5ad7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:08.887707 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4156ee8-793a-4d3c-92ab-f0f1d7aa5ad7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:22:08.889311 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4156ee8-793a-4d3c-92ab-f0f1d7aa5ad7 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:22:08.899967 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1089/2143] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Sep 23 15:22:08 2022] GET /identity/v3/roles => generated 1420 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:22:08.904294 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4156ee8-793a-4d3c-92ab-f0f1d7aa5ad7 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:22:08.915403 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4156ee8-793a-4d3c-92ab-f0f1d7aa5ad7 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:22:08.919694 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd42bdac-cbf3-4b9a-8272-0e80fdef5611 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:22:08.920388 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd42bdac-cbf3-4b9a-8272-0e80fdef5611 admin admin] REQUEST_METHOD: `PUT` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:08.920671 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd42bdac-cbf3-4b9a-8272-0e80fdef5611 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:08.920949 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd42bdac-cbf3-4b9a-8272-0e80fdef5611 admin admin] PATH_INFO: `/v3/projects/130c4dd2e5424c12b0906656c39bc9b6/users/76ed9575994949f58e54209f019a475e/roles/8e2919e79df040e8926f21607818b58c` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:08.922948 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd42bdac-cbf3-4b9a-8272-0e80fdef5611 admin admin] RBAC: Authorizing `identity:create_grant(project_id=130c4dd2e5424c12b0906656c39bc9b6, user_id=76ed9575994949f58e54209f019a475e, role_id=8e2919e79df040e8926f21607818b58c)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:22:08.926201 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd42bdac-cbf3-4b9a-8272-0e80fdef5611 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:22:08.948034 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1090/2144] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Sep 23 15:22:08 2022] PUT /identity/v3/projects/130c4dd2e5424c12b0906656c39bc9b6/users/76ed9575994949f58e54209f019a475e/roles/8e2919e79df040e8926f21607818b58c => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:22:08.959492 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba6d9c2-e335-479f-bd08-e9b159818141 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:08.963678 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba6d9c2-e335-479f-bd08-e9b159818141 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:08.963678 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba6d9c2-e335-479f-bd08-e9b159818141 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:08.998458 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-fba6d9c2-e335-479f-bd08-e9b159818141 None None] MFA Rules not processed for user `76ed9575994949f58e54209f019a475e`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:22:08.998458 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-fba6d9c2-e335-479f-bd08-e9b159818141 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:22:09.004349 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1055/2145] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Sep 23 15:22:08 2022] POST /identity/v3/users => generated 285 bytes in 315 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 23 15:22:09.008367 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafa4c7d-e28b-4ef0-8af9-23a80441ad02 admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:22:09.024766 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafa4c7d-e28b-4ef0-8af9-23a80441ad02 admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:22:09.100230 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1091/2146] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:22:08 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 15:22:09.101303 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba6d9c2-e335-479f-bd08-e9b159818141 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:22:09.103030 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-fba6d9c2-e335-479f-bd08-e9b159818141 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:22:09.109743 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f06e36-2db3-4db8-ad99-1ea2be3aa121 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:22:09.110758 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f06e36-2db3-4db8-ad99-1ea2be3aa121 admin admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:09.111651 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f06e36-2db3-4db8-ad99-1ea2be3aa121 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:09.112059 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f06e36-2db3-4db8-ad99-1ea2be3aa121 admin admin] PATH_INFO: `/v3/roles` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:09.112847 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f06e36-2db3-4db8-ad99-1ea2be3aa121 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:22:09.114695 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f06e36-2db3-4db8-ad99-1ea2be3aa121 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:22:09.120138 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba6d9c2-e335-479f-bd08-e9b159818141 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:22:09.126187 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8111b36b-0bbc-4b44-b18f-e132ac932cf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:22:09.126790 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8111b36b-0bbc-4b44-b18f-e132ac932cf4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:09.127242 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8111b36b-0bbc-4b44-b18f-e132ac932cf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:09.127891 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8111b36b-0bbc-4b44-b18f-e132ac932cf4 admin admin] PATH_INFO: `/v3/users/a3430eb5e7104dc2b3a1c3500614506e` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:09.129102 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8111b36b-0bbc-4b44-b18f-e132ac932cf4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a3430eb5e7104dc2b3a1c3500614506e)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:22:09.131210 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8111b36b-0bbc-4b44-b18f-e132ac932cf4 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:22:09.136372 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1056/2147] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Sep 23 15:22:09 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:22:09.136372 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f06e36-2db3-4db8-ad99-1ea2be3aa121 admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:22:09.139458 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-b8f06e36-2db3-4db8-ad99-1ea2be3aa121 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:22:09.201736 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f06e36-2db3-4db8-ad99-1ea2be3aa121 admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:22:09.206414 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adb0e64-12ad-498b-a9db-e2b8dcc4c4a1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feca6278be904ef8b0dc580a67e477f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:22:09.207039 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adb0e64-12ad-498b-a9db-e2b8dcc4c4a1 service cinder] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:09.207327 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adb0e64-12ad-498b-a9db-e2b8dcc4c4a1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:09.207600 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adb0e64-12ad-498b-a9db-e2b8dcc4c4a1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:09.218880 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3adb0e64-12ad-498b-a9db-e2b8dcc4c4a1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:22:09.221459 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3adb0e64-12ad-498b-a9db-e2b8dcc4c4a1 service cinder] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:22:09.232573 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1057/2148] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:22:09 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:22:09.233276 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adb0e64-12ad-498b-a9db-e2b8dcc4c4a1 service cinder] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:22:09.244320 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adb0e64-12ad-498b-a9db-e2b8dcc4c4a1 service cinder] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:22:09.249029 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7420e1-bd10-44b1-96f4-20899c191871 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:22:09.249623 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7420e1-bd10-44b1-96f4-20899c191871 admin admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:09.249894 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7420e1-bd10-44b1-96f4-20899c191871 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:09.250180 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7420e1-bd10-44b1-96f4-20899c191871 admin admin] PATH_INFO: `/v3/roles` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:09.250842 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7420e1-bd10-44b1-96f4-20899c191871 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:22:09.252502 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7420e1-bd10-44b1-96f4-20899c191871 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:22:09.263393 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1058/2149] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Sep 23 15:22:09 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:22:09.268809 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7420e1-bd10-44b1-96f4-20899c191871 admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:22:09.288413 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7420e1-bd10-44b1-96f4-20899c191871 admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:22:09.293966 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e204668-9f43-4df9-a183-c7d71a58677e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:22:09.294771 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e204668-9f43-4df9-a183-c7d71a58677e admin admin] REQUEST_METHOD: `PUT` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:09.295158 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e204668-9f43-4df9-a183-c7d71a58677e admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:09.295509 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e204668-9f43-4df9-a183-c7d71a58677e admin admin] PATH_INFO: `/v3/projects/5303d4f32d4c458aba3340b06312cc46/users/ffd24319a3884c0c965fc611db7d40ae/roles/af257be2dee54d6b942f58650196582c` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:09.313069 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e204668-9f43-4df9-a183-c7d71a58677e admin admin] RBAC: Authorizing `identity:create_grant(project_id=5303d4f32d4c458aba3340b06312cc46, user_id=ffd24319a3884c0c965fc611db7d40ae, role_id=af257be2dee54d6b942f58650196582c)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:22:09.316881 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e204668-9f43-4df9-a183-c7d71a58677e admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:22:09.418377 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1059/2150] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Sep 23 15:22:09 2022] PUT /identity/v3/projects/5303d4f32d4c458aba3340b06312cc46/users/ffd24319a3884c0c965fc611db7d40ae/roles/af257be2dee54d6b942f58650196582c => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 15:22:09.430025 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4cd690-bc31-4e9d-8eb1-e35e6321ab66 None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:09.430303 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4cd690-bc31-4e9d-8eb1-e35e6321ab66 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:09.430582 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4cd690-bc31-4e9d-8eb1-e35e6321ab66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:09.435005 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-8111b36b-0bbc-4b44-b18f-e132ac932cf4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a3430eb5e7104dc2b3a1c3500614506e'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:22:09.448298 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-8111b36b-0bbc-4b44-b18f-e132ac932cf4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a3430eb5e7104dc2b3a1c3500614506e'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:22:09.456475 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-8111b36b-0bbc-4b44-b18f-e132ac932cf4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a3430eb5e7104dc2b3a1c3500614506e'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:22:09.461991 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-6c4cd690-bc31-4e9d-8eb1-e35e6321ab66 None None] MFA Rules not processed for user `ffd24319a3884c0c965fc611db7d40ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:22:09.463199 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-6c4cd690-bc31-4e9d-8eb1-e35e6321ab66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:22:09.543110 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-8111b36b-0bbc-4b44-b18f-e132ac932cf4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a3430eb5e7104dc2b3a1c3500614506e'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:22:09.543324 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1060/2151] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:22:09 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 15:22:09.557632 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-8111b36b-0bbc-4b44-b18f-e132ac932cf4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a3430eb5e7104dc2b3a1c3500614506e'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:22:09.557726 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4cd690-bc31-4e9d-8eb1-e35e6321ab66 None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:22:09.558632 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-6c4cd690-bc31-4e9d-8eb1-e35e6321ab66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:22:09.560403 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1092/2152] 10.222.0.55 () {58 vars in 1224 bytes} [Fri Sep 23 15:22:09 2022] DELETE /identity/v3/users/a3430eb5e7104dc2b3a1c3500614506e => generated 0 bytes in 457 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:22:09.562065 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039ea2c8-9ed2-4b86-93f5-0e9c87cdde67 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:09.562602 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039ea2c8-9ed2-4b86-93f5-0e9c87cdde67 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:09.564797 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039ea2c8-9ed2-4b86-93f5-0e9c87cdde67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:09.589152 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-039ea2c8-9ed2-4b86-93f5-0e9c87cdde67 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:22:09.597342 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-039ea2c8-9ed2-4b86-93f5-0e9c87cdde67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:22:09.667678 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:22:09.675584 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4cd690-bc31-4e9d-8eb1-e35e6321ab66 None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:22:09.752604 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcbffab-6fbe-47a1-947c-135aa2c4f733 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51c39f1d05634589a88c0b6477bd014e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:22:09.753305 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcbffab-6fbe-47a1-947c-135aa2c4f733 service glance] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:09.753614 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcbffab-6fbe-47a1-947c-135aa2c4f733 service glance] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:09.753890 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcbffab-6fbe-47a1-947c-135aa2c4f733 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:09.758795 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-0dcbffab-6fbe-47a1-947c-135aa2c4f733 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:22:09.775829 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1093/2153] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:22:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:22:09.775829 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039ea2c8-9ed2-4b86-93f5-0e9c87cdde67 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:22:09.775829 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-039ea2c8-9ed2-4b86-93f5-0e9c87cdde67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:22:09.828556 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dcbffab-6fbe-47a1-947c-135aa2c4f733 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:22:09.828849 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039ea2c8-9ed2-4b86-93f5-0e9c87cdde67 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:22:09.833120 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f42d85-6353-4fb8-909f-9eaaf78e5b91 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feca6278be904ef8b0dc580a67e477f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:22:09.833572 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dcbffab-6fbe-47a1-947c-135aa2c4f733 service glance] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:22:09.833747 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f42d85-6353-4fb8-909f-9eaaf78e5b91 service cinder] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:09.834020 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f42d85-6353-4fb8-909f-9eaaf78e5b91 service cinder] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:09.834424 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f42d85-6353-4fb8-909f-9eaaf78e5b91 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:09.836507 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-e4f42d85-6353-4fb8-909f-9eaaf78e5b91 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:22:09.846050 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1061/2154] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:22:09 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:22:09.846877 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcbffab-6fbe-47a1-947c-135aa2c4f733 service glance] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:22:09.848370 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-0dcbffab-6fbe-47a1-947c-135aa2c4f733 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:22:09.920574 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f42d85-6353-4fb8-909f-9eaaf78e5b91 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:22:09.923855 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f42d85-6353-4fb8-909f-9eaaf78e5b91 service cinder] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:22:09.937393 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1094/2155] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:22:09 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:22:09.940773 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcbffab-6fbe-47a1-947c-135aa2c4f733 service glance] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:22:09.944899 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb260aea-8dba-4eaf-a5b2-98a4c23f4211 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd259178570864ffeb936ee10dfec6b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:22:09.945398 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb260aea-8dba-4eaf-a5b2-98a4c23f4211 service neutron] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:09.945727 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb260aea-8dba-4eaf-a5b2-98a4c23f4211 service neutron] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:09.946063 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb260aea-8dba-4eaf-a5b2-98a4c23f4211 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:09.956485 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb260aea-8dba-4eaf-a5b2-98a4c23f4211 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:22:09.957781 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb260aea-8dba-4eaf-a5b2-98a4c23f4211 service neutron] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:22:09.968415 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1062/2156] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:22:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:22:10.037951 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:22:10.530934 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f42d85-6353-4fb8-909f-9eaaf78e5b91 service cinder] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:22:10.532986 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-e4f42d85-6353-4fb8-909f-9eaaf78e5b91 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:22:10.551300 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f42d85-6353-4fb8-909f-9eaaf78e5b91 service cinder] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:22:10.556440 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c81996e-3903-412c-85a6-28e2b3b743a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:22:10.557096 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c81996e-3903-412c-85a6-28e2b3b743a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:10.557535 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c81996e-3903-412c-85a6-28e2b3b743a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:10.557812 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c81996e-3903-412c-85a6-28e2b3b743a6 admin admin] PATH_INFO: `/v3/projects/70a4b13673f94eca9e57598dd454dd87` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:10.559035 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c81996e-3903-412c-85a6-28e2b3b743a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=70a4b13673f94eca9e57598dd454dd87)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:22:10.561077 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c81996e-3903-412c-85a6-28e2b3b743a6 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:22:10.796521 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-6c81996e-3903-412c-85a6-28e2b3b743a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '70a4b13673f94eca9e57598dd454dd87', 'request_id': 'req-6c81996e-3903-412c-85a6-28e2b3b743a6'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:22:10.804272 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-6c81996e-3903-412c-85a6-28e2b3b743a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '70a4b13673f94eca9e57598dd454dd87', 'request_id': 'req-6c81996e-3903-412c-85a6-28e2b3b743a6'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:22:10.813039 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-6c81996e-3903-412c-85a6-28e2b3b743a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '70a4b13673f94eca9e57598dd454dd87', 'request_id': 'req-6c81996e-3903-412c-85a6-28e2b3b743a6'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:22:10.950018 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-6c81996e-3903-412c-85a6-28e2b3b743a6 admin admin] The token cache is being invalidate because project 70a4b13673f94eca9e57598dd454dd87 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81994) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 15:22:10.950326 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-6c81996e-3903-412c-85a6-28e2b3b743a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:22:10.956531 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-6c81996e-3903-412c-85a6-28e2b3b743a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:22:10.964779 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1095/2157] 10.222.0.55 () {58 vars in 1233 bytes} [Fri Sep 23 15:22:10 2022] DELETE /identity/v3/projects/70a4b13673f94eca9e57598dd454dd87 => generated 0 bytes in 431 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:22:10.966887 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb260aea-8dba-4eaf-a5b2-98a4c23f4211 service neutron] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:22:11.017318 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb260aea-8dba-4eaf-a5b2-98a4c23f4211 service neutron] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:22:11.118037 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1ed8ef-8f7e-4ff0-a3b5-16119f026bd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:22:11.118665 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1ed8ef-8f7e-4ff0-a3b5-16119f026bd6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:11.118973 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1ed8ef-8f7e-4ff0-a3b5-16119f026bd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:11.119254 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1ed8ef-8f7e-4ff0-a3b5-16119f026bd6 admin admin] PATH_INFO: `/v3/users/f2c57745fb374059a028a24d2ee57576` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:11.120411 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1ed8ef-8f7e-4ff0-a3b5-16119f026bd6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f2c57745fb374059a028a24d2ee57576)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:22:11.122348 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1ed8ef-8f7e-4ff0-a3b5-16119f026bd6 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:22:11.464799 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-9f1ed8ef-8f7e-4ff0-a3b5-16119f026bd6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f2c57745fb374059a028a24d2ee57576'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:22:11.507418 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-9f1ed8ef-8f7e-4ff0-a3b5-16119f026bd6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f2c57745fb374059a028a24d2ee57576'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:22:11.507418 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-9f1ed8ef-8f7e-4ff0-a3b5-16119f026bd6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f2c57745fb374059a028a24d2ee57576'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:22:11.643071 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-9f1ed8ef-8f7e-4ff0-a3b5-16119f026bd6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f2c57745fb374059a028a24d2ee57576'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:22:11.643699 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-9f1ed8ef-8f7e-4ff0-a3b5-16119f026bd6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f2c57745fb374059a028a24d2ee57576'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:22:11.651037 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1063/2158] 10.222.0.55 () {58 vars in 1224 bytes} [Fri Sep 23 15:22:10 2022] DELETE /identity/v3/users/f2c57745fb374059a028a24d2ee57576 => generated 0 bytes in 685 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:22:11.736491 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=89,keystone:INSERT=2,keystone:DELETE=9 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:22:12.292093 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c81996e-3903-412c-85a6-28e2b3b743a6 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:22:12.298778 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-6c81996e-3903-412c-85a6-28e2b3b743a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:22:12.331563 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c81996e-3903-412c-85a6-28e2b3b743a6 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:22:12.402135 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02007217-2ce1-4edb-ad45-a72cdf6bcaa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:22:12.403340 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02007217-2ce1-4edb-ad45-a72cdf6bcaa1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:12.403809 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02007217-2ce1-4edb-ad45-a72cdf6bcaa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:12.404336 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02007217-2ce1-4edb-ad45-a72cdf6bcaa1 admin admin] PATH_INFO: `/v3/projects/ff892905862b43eabe4699507af00b6c` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:12.406350 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02007217-2ce1-4edb-ad45-a72cdf6bcaa1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ff892905862b43eabe4699507af00b6c)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:22:12.411646 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02007217-2ce1-4edb-ad45-a72cdf6bcaa1 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:22:12.426424 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:22:12.586206 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-02007217-2ce1-4edb-ad45-a72cdf6bcaa1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ff892905862b43eabe4699507af00b6c', 'request_id': 'req-02007217-2ce1-4edb-ad45-a72cdf6bcaa1'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:22:12.594281 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-02007217-2ce1-4edb-ad45-a72cdf6bcaa1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ff892905862b43eabe4699507af00b6c', 'request_id': 'req-02007217-2ce1-4edb-ad45-a72cdf6bcaa1'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:22:12.603127 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-02007217-2ce1-4edb-ad45-a72cdf6bcaa1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ff892905862b43eabe4699507af00b6c', 'request_id': 'req-02007217-2ce1-4edb-ad45-a72cdf6bcaa1'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:22:12.706123 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-02007217-2ce1-4edb-ad45-a72cdf6bcaa1 admin admin] The token cache is being invalidate because project ff892905862b43eabe4699507af00b6c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81994) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 15:22:12.706447 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-02007217-2ce1-4edb-ad45-a72cdf6bcaa1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:22:12.707102 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-02007217-2ce1-4edb-ad45-a72cdf6bcaa1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:22:12.712749 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1096/2159] 10.222.0.55 () {58 vars in 1233 bytes} [Fri Sep 23 15:22:12 2022] DELETE /identity/v3/projects/ff892905862b43eabe4699507af00b6c => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:22:12.713909 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1603a6ae-3693-4db7-9f70-01ef1c42ba37 None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:12.714216 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1603a6ae-3693-4db7-9f70-01ef1c42ba37 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:12.714495 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1603a6ae-3693-4db7-9f70-01ef1c42ba37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:12.742247 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-1603a6ae-3693-4db7-9f70-01ef1c42ba37 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:22:12.742247 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-1603a6ae-3693-4db7-9f70-01ef1c42ba37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:22:12.811739 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1064/2160] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:22:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:22:12.820591 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc0635e-6f5e-405f-8270-23eb2e256902 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:12.820924 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc0635e-6f5e-405f-8270-23eb2e256902 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:12.821205 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc0635e-6f5e-405f-8270-23eb2e256902 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:12.843305 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-fbc0635e-6f5e-405f-8270-23eb2e256902 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:22:12.844727 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-fbc0635e-6f5e-405f-8270-23eb2e256902 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:22:12.858953 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1097/2161] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:22:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:22:12.861456 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1603a6ae-3693-4db7-9f70-01ef1c42ba37 None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:22:12.872467 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1603a6ae-3693-4db7-9f70-01ef1c42ba37 None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:22:12.878949 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac51c0f-cbbc-4487-bc44-2e0fdcee918f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:22:12.879096 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac51c0f-cbbc-4487-bc44-2e0fdcee918f admin admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:12.879293 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac51c0f-cbbc-4487-bc44-2e0fdcee918f admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:12.879753 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac51c0f-cbbc-4487-bc44-2e0fdcee918f admin admin] PATH_INFO: `/v3/domains` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:12.880450 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac51c0f-cbbc-4487-bc44-2e0fdcee918f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:22:12.881362 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac51c0f-cbbc-4487-bc44-2e0fdcee918f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:22:12.882516 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac51c0f-cbbc-4487-bc44-2e0fdcee918f admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:22:12.912613 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1065/2162] 10.222.0.55 () {58 vars in 1153 bytes} [Fri Sep 23 15:22:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:22:12.915840 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc0635e-6f5e-405f-8270-23eb2e256902 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:22:12.928784 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc0635e-6f5e-405f-8270-23eb2e256902 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:22:12.948403 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a6aba3-b3bc-4b97-8d44-a75bb3d7fad0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:22:12.950307 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a6aba3-b3bc-4b97-8d44-a75bb3d7fad0 admin admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:12.950841 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a6aba3-b3bc-4b97-8d44-a75bb3d7fad0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:12.951163 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a6aba3-b3bc-4b97-8d44-a75bb3d7fad0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:12.959490 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a6aba3-b3bc-4b97-8d44-a75bb3d7fad0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:22:12.959677 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a6aba3-b3bc-4b97-8d44-a75bb3d7fad0 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:22:13.113425 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1098/2163] 10.222.0.55 () {60 vars in 1153 bytes} [Fri Sep 23 15:22:12 2022] POST /identity/v3/projects => generated 375 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:22:13.114520 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac51c0f-cbbc-4487-bc44-2e0fdcee918f admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:22:13.125125 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac51c0f-cbbc-4487-bc44-2e0fdcee918f admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:22:13.203308 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c37f956-4509-4040-b63a-8f2f411cdc48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:22:13.203970 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c37f956-4509-4040-b63a-8f2f411cdc48 admin admin] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:13.204286 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c37f956-4509-4040-b63a-8f2f411cdc48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:13.204734 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c37f956-4509-4040-b63a-8f2f411cdc48 admin admin] PATH_INFO: `/v3/users` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:13.206224 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c37f956-4509-4040-b63a-8f2f411cdc48 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:22:13.207610 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c37f956-4509-4040-b63a-8f2f411cdc48 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:22:13.354546 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1066/2164] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Sep 23 15:22:13 2022] POST /identity/v3/users => generated 288 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:22:13.358523 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a6aba3-b3bc-4b97-8d44-a75bb3d7fad0 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:22:13.374977 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a6aba3-b3bc-4b97-8d44-a75bb3d7fad0 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:22:13.381261 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45402199-1f24-46cb-946b-f13e2fb31162 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:22:13.382264 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45402199-1f24-46cb-946b-f13e2fb31162 admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:13.382560 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45402199-1f24-46cb-946b-f13e2fb31162 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:13.382831 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45402199-1f24-46cb-946b-f13e2fb31162 admin admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:13.383746 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45402199-1f24-46cb-946b-f13e2fb31162 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:22:13.386212 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45402199-1f24-46cb-946b-f13e2fb31162 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:22:13.398951 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1099/2165] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Sep 23 15:22:13 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:22:13.402700 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c37f956-4509-4040-b63a-8f2f411cdc48 admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:22:13.416378 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c37f956-4509-4040-b63a-8f2f411cdc48 admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:22:13.424930 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b17f3f-136f-4d10-bd1e-f6e1fa2a25dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:22:13.425981 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b17f3f-136f-4d10-bd1e-f6e1fa2a25dd admin admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:13.426487 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b17f3f-136f-4d10-bd1e-f6e1fa2a25dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:13.427056 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b17f3f-136f-4d10-bd1e-f6e1fa2a25dd admin admin] PATH_INFO: `/v3/roles` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:13.428028 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b17f3f-136f-4d10-bd1e-f6e1fa2a25dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:22:13.433991 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b17f3f-136f-4d10-bd1e-f6e1fa2a25dd admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:22:13.444916 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1067/2166] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Sep 23 15:22:13 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:22:13.450166 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45402199-1f24-46cb-946b-f13e2fb31162 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:22:13.460961 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45402199-1f24-46cb-946b-f13e2fb31162 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:22:13.465829 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b604d80b-4731-455e-9c68-b9711cfa8b79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:22:13.466561 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b604d80b-4731-455e-9c68-b9711cfa8b79 admin admin] REQUEST_METHOD: `PUT` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:13.467010 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b604d80b-4731-455e-9c68-b9711cfa8b79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:13.467358 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b604d80b-4731-455e-9c68-b9711cfa8b79 admin admin] PATH_INFO: `/v3/projects/5ff374b0f9b640009a325a14d202a6a8/users/664fc815c345413a95a9b3f38c976274/roles/af257be2dee54d6b942f58650196582c` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:13.495809 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b604d80b-4731-455e-9c68-b9711cfa8b79 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ff374b0f9b640009a325a14d202a6a8, user_id=664fc815c345413a95a9b3f38c976274, role_id=af257be2dee54d6b942f58650196582c)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:22:13.502729 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b604d80b-4731-455e-9c68-b9711cfa8b79 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:22:13.590136 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1100/2167] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Sep 23 15:22:13 2022] PUT /identity/v3/projects/5ff374b0f9b640009a325a14d202a6a8/users/664fc815c345413a95a9b3f38c976274/roles/af257be2dee54d6b942f58650196582c => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:22:13.607390 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc2c1f1-6a5b-4d4f-9cbd-d74bb36258e9 None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:13.607681 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc2c1f1-6a5b-4d4f-9cbd-d74bb36258e9 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:13.607957 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc2c1f1-6a5b-4d4f-9cbd-d74bb36258e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:13.681900 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-efc2c1f1-6a5b-4d4f-9cbd-d74bb36258e9 None None] MFA Rules not processed for user `664fc815c345413a95a9b3f38c976274`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:22:13.683151 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-efc2c1f1-6a5b-4d4f-9cbd-d74bb36258e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:22:13.781168 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1068/2168] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:22:13 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:22:13.794641 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b604d80b-4731-455e-9c68-b9711cfa8b79 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:22:13.796584 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-b604d80b-4731-455e-9c68-b9711cfa8b79 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:22:13.869287 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b604d80b-4731-455e-9c68-b9711cfa8b79 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:22:13.873208 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf955c75-e4aa-49c2-8822-e540f281ab97 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feca6278be904ef8b0dc580a67e477f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:22:13.873999 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf955c75-e4aa-49c2-8822-e540f281ab97 service cinder] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:13.874460 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf955c75-e4aa-49c2-8822-e540f281ab97 service cinder] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:13.874884 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf955c75-e4aa-49c2-8822-e540f281ab97 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:13.892377 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf955c75-e4aa-49c2-8822-e540f281ab97 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:22:13.892377 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf955c75-e4aa-49c2-8822-e540f281ab97 service cinder] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:22:13.900768 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1101/2169] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:22:13 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:22:15.206154 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=81,keystone:INSERT=2,keystone:DELETE=17 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:22:16.027631 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=97,keystone:INSERT=2,keystone:DELETE=1 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:22:16.351782 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1e4d00-0613-4279-9d0f-7d7f81fbdcb7 None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:16.352307 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1e4d00-0613-4279-9d0f-7d7f81fbdcb7 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:16.352620 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1e4d00-0613-4279-9d0f-7d7f81fbdcb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:16.407348 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-3d1e4d00-0613-4279-9d0f-7d7f81fbdcb7 None None] MFA Rules not processed for user `4bdaecd5dd5d461383878c4b96ef98be`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:22:16.408851 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-3d1e4d00-0613-4279-9d0f-7d7f81fbdcb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:22:16.487255 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1069/2170] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:22:16 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:22:16.491834 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf955c75-e4aa-49c2-8822-e540f281ab97 service cinder] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:22:16.493403 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-cf955c75-e4aa-49c2-8822-e540f281ab97 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:22:16.569764 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf955c75-e4aa-49c2-8822-e540f281ab97 service cinder] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:22:16.577539 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3f6c94-496d-4e02-b88e-cbe77ccca018 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:22:16.578493 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3f6c94-496d-4e02-b88e-cbe77ccca018 admin admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:16.579447 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3f6c94-496d-4e02-b88e-cbe77ccca018 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:16.579725 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3f6c94-496d-4e02-b88e-cbe77ccca018 admin admin] PATH_INFO: `/v3/projects` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:16.580794 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3f6c94-496d-4e02-b88e-cbe77ccca018 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:22:16.582772 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3f6c94-496d-4e02-b88e-cbe77ccca018 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:22:16.697625 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1102/2171] 10.222.0.55 () {60 vars in 1153 bytes} [Fri Sep 23 15:22:16 2022] POST /identity/v3/projects => generated 371 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:22:16.701787 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1e4d00-0613-4279-9d0f-7d7f81fbdcb7 None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:22:16.713282 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3f6c94-496d-4e02-b88e-cbe77ccca018 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:22:16.714657 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1e4d00-0613-4279-9d0f-7d7f81fbdcb7 None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:22:16.716223 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-4c3f6c94-496d-4e02-b88e-cbe77ccca018 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:22:16.799509 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7e0bbb-6c4e-4674-bcda-35635c955ae6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:22:16.800157 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7e0bbb-6c4e-4674-bcda-35635c955ae6 admin admin] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:16.800261 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3f6c94-496d-4e02-b88e-cbe77ccca018 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:22:16.800445 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7e0bbb-6c4e-4674-bcda-35635c955ae6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:16.800722 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7e0bbb-6c4e-4674-bcda-35635c955ae6 admin admin] PATH_INFO: `/v3/users` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:16.801361 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7e0bbb-6c4e-4674-bcda-35635c955ae6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:22:16.803294 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7e0bbb-6c4e-4674-bcda-35635c955ae6 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:22:16.804998 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42364d06-5649-45a7-875a-5d5d0ede692f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:22:16.807516 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42364d06-5649-45a7-875a-5d5d0ede692f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:16.807516 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42364d06-5649-45a7-875a-5d5d0ede692f admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:16.807516 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42364d06-5649-45a7-875a-5d5d0ede692f admin admin] PATH_INFO: `/v3/users/ae62db6a9d164a2a8a9602b8c544c0b0` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:16.807516 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42364d06-5649-45a7-875a-5d5d0ede692f admin admin] RBAC: Authorizing `identity:delete_user(user_id=ae62db6a9d164a2a8a9602b8c544c0b0)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:22:16.809302 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42364d06-5649-45a7-875a-5d5d0ede692f admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:22:16.980009 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1070/2172] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Sep 23 15:22:16 2022] POST /identity/v3/users => generated 292 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:22:16.984401 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7e0bbb-6c4e-4674-bcda-35635c955ae6 admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:22:16.998221 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7e0bbb-6c4e-4674-bcda-35635c955ae6 admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:22:17.004629 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7e0896-5d28-4fc4-82c3-0d867264843e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:22:17.008252 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7e0896-5d28-4fc4-82c3-0d867264843e admin admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:17.008252 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7e0896-5d28-4fc4-82c3-0d867264843e admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:17.008252 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7e0896-5d28-4fc4-82c3-0d867264843e admin admin] PATH_INFO: `/v3/roles` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:17.008252 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7e0896-5d28-4fc4-82c3-0d867264843e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:22:17.008252 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7e0896-5d28-4fc4-82c3-0d867264843e admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:22:17.020095 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1071/2173] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Sep 23 15:22:16 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 23 15:22:17.023839 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7e0896-5d28-4fc4-82c3-0d867264843e admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:22:17.035334 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7e0896-5d28-4fc4-82c3-0d867264843e admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:22:17.041229 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1954cd3-38e6-48db-b68f-c4fc002dae5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:22:17.042206 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1954cd3-38e6-48db-b68f-c4fc002dae5d admin admin] REQUEST_METHOD: `PUT` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:17.042823 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1954cd3-38e6-48db-b68f-c4fc002dae5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:17.043044 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1954cd3-38e6-48db-b68f-c4fc002dae5d admin admin] PATH_INFO: `/v3/projects/ce652035804f47dc803c040c20704414/users/3a349a48d97d4d0283d630c55b36bd18/roles/8e2919e79df040e8926f21607818b58c` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:17.060946 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1954cd3-38e6-48db-b68f-c4fc002dae5d admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce652035804f47dc803c040c20704414, user_id=3a349a48d97d4d0283d630c55b36bd18, role_id=8e2919e79df040e8926f21607818b58c)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:22:17.065400 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1954cd3-38e6-48db-b68f-c4fc002dae5d admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:22:17.139420 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-42364d06-5649-45a7-875a-5d5d0ede692f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ae62db6a9d164a2a8a9602b8c544c0b0'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:22:17.158860 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-42364d06-5649-45a7-875a-5d5d0ede692f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ae62db6a9d164a2a8a9602b8c544c0b0'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:22:17.178359 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-42364d06-5649-45a7-875a-5d5d0ede692f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ae62db6a9d164a2a8a9602b8c544c0b0'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:22:17.205262 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1072/2174] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Sep 23 15:22:17 2022] PUT /identity/v3/projects/ce652035804f47dc803c040c20704414/users/3a349a48d97d4d0283d630c55b36bd18/roles/8e2919e79df040e8926f21607818b58c => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 15:22:17.209128 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1954cd3-38e6-48db-b68f-c4fc002dae5d admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:22:17.276994 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1954cd3-38e6-48db-b68f-c4fc002dae5d admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:22:17.277564 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a877139-dae8-4e27-adce-5dab53cfff5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:22:17.278177 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a877139-dae8-4e27-adce-5dab53cfff5a admin admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:17.278457 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a877139-dae8-4e27-adce-5dab53cfff5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:17.278732 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a877139-dae8-4e27-adce-5dab53cfff5a admin admin] PATH_INFO: `/v3/roles` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:17.279376 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a877139-dae8-4e27-adce-5dab53cfff5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:22:17.281118 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a877139-dae8-4e27-adce-5dab53cfff5a admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:22:17.294114 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1073/2175] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Sep 23 15:22:17 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 23 15:22:17.296336 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a877139-dae8-4e27-adce-5dab53cfff5a admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:22:17.308026 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a877139-dae8-4e27-adce-5dab53cfff5a admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:22:17.314283 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497db337-9825-4f08-9acb-c3082d8f7e00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:22:17.316554 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497db337-9825-4f08-9acb-c3082d8f7e00 admin admin] REQUEST_METHOD: `PUT` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:17.316554 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497db337-9825-4f08-9acb-c3082d8f7e00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:17.316554 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497db337-9825-4f08-9acb-c3082d8f7e00 admin admin] PATH_INFO: `/v3/projects/ce652035804f47dc803c040c20704414/users/3a349a48d97d4d0283d630c55b36bd18/roles/8e2919e79df040e8926f21607818b58c` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:17.317842 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497db337-9825-4f08-9acb-c3082d8f7e00 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce652035804f47dc803c040c20704414, user_id=3a349a48d97d4d0283d630c55b36bd18, role_id=8e2919e79df040e8926f21607818b58c)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:22:17.320857 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497db337-9825-4f08-9acb-c3082d8f7e00 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:22:17.331525 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-42364d06-5649-45a7-875a-5d5d0ede692f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ae62db6a9d164a2a8a9602b8c544c0b0'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:22:17.332533 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-42364d06-5649-45a7-875a-5d5d0ede692f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ae62db6a9d164a2a8a9602b8c544c0b0'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:22:17.333646 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1074/2176] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Sep 23 15:22:17 2022] PUT /identity/v3/projects/ce652035804f47dc803c040c20704414/users/3a349a48d97d4d0283d630c55b36bd18/roles/8e2919e79df040e8926f21607818b58c => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 15:22:17.347079 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1103/2177] 10.222.0.55 () {58 vars in 1224 bytes} [Fri Sep 23 15:22:16 2022] DELETE /identity/v3/users/ae62db6a9d164a2a8a9602b8c544c0b0 => generated 0 bytes in 631 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 15:22:17.350087 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87236d5-5d69-40c0-b864-ac6e45c436b8 None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:17.350507 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87236d5-5d69-40c0-b864-ac6e45c436b8 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:17.350859 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87236d5-5d69-40c0-b864-ac6e45c436b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:17.378916 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-d87236d5-5d69-40c0-b864-ac6e45c436b8 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:22:17.380745 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-d87236d5-5d69-40c0-b864-ac6e45c436b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:22:17.464712 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1075/2178] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:22:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:22:17.474866 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42364d06-5649-45a7-875a-5d5d0ede692f admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:22:17.476747 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-42364d06-5649-45a7-875a-5d5d0ede692f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:22:17.540800 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42364d06-5649-45a7-875a-5d5d0ede692f admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:22:17.545536 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b3460d-4bb8-465d-b017-afefa6620c2a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd259178570864ffeb936ee10dfec6b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:22:17.546310 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b3460d-4bb8-465d-b017-afefa6620c2a service neutron] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:17.546597 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b3460d-4bb8-465d-b017-afefa6620c2a service neutron] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:17.547125 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b3460d-4bb8-465d-b017-afefa6620c2a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:17.557844 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b3460d-4bb8-465d-b017-afefa6620c2a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:22:17.559539 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b3460d-4bb8-465d-b017-afefa6620c2a service neutron] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:22:17.573007 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1104/2179] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:22:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:22:18.317237 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87236d5-5d69-40c0-b864-ac6e45c436b8 None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:22:18.318774 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-d87236d5-5d69-40c0-b864-ac6e45c436b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:22:18.337812 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87236d5-5d69-40c0-b864-ac6e45c436b8 None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:22:18.344912 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997ca721-1af5-4de0-be16-5aaa5252ba04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:22:18.345645 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997ca721-1af5-4de0-be16-5aaa5252ba04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:18.345926 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997ca721-1af5-4de0-be16-5aaa5252ba04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:18.346199 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997ca721-1af5-4de0-be16-5aaa5252ba04 admin admin] PATH_INFO: `/v3/projects/4e8b30906b244e4fb7bcd5da9b8fb279` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:18.347371 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997ca721-1af5-4de0-be16-5aaa5252ba04 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4e8b30906b244e4fb7bcd5da9b8fb279)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:22:18.349297 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997ca721-1af5-4de0-be16-5aaa5252ba04 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:22:18.530575 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-997ca721-1af5-4de0-be16-5aaa5252ba04 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4e8b30906b244e4fb7bcd5da9b8fb279', 'request_id': 'req-997ca721-1af5-4de0-be16-5aaa5252ba04'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:22:18.537992 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-997ca721-1af5-4de0-be16-5aaa5252ba04 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4e8b30906b244e4fb7bcd5da9b8fb279', 'request_id': 'req-997ca721-1af5-4de0-be16-5aaa5252ba04'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:22:18.659437 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-997ca721-1af5-4de0-be16-5aaa5252ba04 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4e8b30906b244e4fb7bcd5da9b8fb279', 'request_id': 'req-997ca721-1af5-4de0-be16-5aaa5252ba04'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:22:18.660035 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:22:18.676327 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-997ca721-1af5-4de0-be16-5aaa5252ba04 admin admin] The token cache is being invalidate because project 4e8b30906b244e4fb7bcd5da9b8fb279 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81993) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 15:22:18.676327 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-997ca721-1af5-4de0-be16-5aaa5252ba04 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:22:18.676327 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-997ca721-1af5-4de0-be16-5aaa5252ba04 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:22:18.678281 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1076/2180] 10.222.0.55 () {58 vars in 1233 bytes} [Fri Sep 23 15:22:18 2022] DELETE /identity/v3/projects/4e8b30906b244e4fb7bcd5da9b8fb279 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:22:18.682240 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b3460d-4bb8-465d-b017-afefa6620c2a service neutron] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:22:18.693409 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b3460d-4bb8-465d-b017-afefa6620c2a service neutron] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:22:18.765378 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d71a8e0-b119-4e63-84b7-27ecc207edfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:22:18.766574 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d71a8e0-b119-4e63-84b7-27ecc207edfa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:18.766861 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d71a8e0-b119-4e63-84b7-27ecc207edfa admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:18.767140 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d71a8e0-b119-4e63-84b7-27ecc207edfa admin admin] PATH_INFO: `/v3/users/76ed9575994949f58e54209f019a475e` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:18.768585 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d71a8e0-b119-4e63-84b7-27ecc207edfa admin admin] RBAC: Authorizing `identity:delete_user(user_id=76ed9575994949f58e54209f019a475e)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:22:18.770587 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d71a8e0-b119-4e63-84b7-27ecc207edfa admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:22:19.151050 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-6d71a8e0-b119-4e63-84b7-27ecc207edfa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '76ed9575994949f58e54209f019a475e'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:22:19.164102 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-6d71a8e0-b119-4e63-84b7-27ecc207edfa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '76ed9575994949f58e54209f019a475e'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:22:19.177679 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-6d71a8e0-b119-4e63-84b7-27ecc207edfa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '76ed9575994949f58e54209f019a475e'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:22:19.346236 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-6d71a8e0-b119-4e63-84b7-27ecc207edfa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '76ed9575994949f58e54209f019a475e'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:22:19.346948 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-6d71a8e0-b119-4e63-84b7-27ecc207edfa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '76ed9575994949f58e54209f019a475e'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:22:19.361048 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1105/2181] 10.222.0.55 () {58 vars in 1224 bytes} [Fri Sep 23 15:22:18 2022] DELETE /identity/v3/users/76ed9575994949f58e54209f019a475e => generated 0 bytes in 677 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:22:19.581539 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:22:20.097252 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997ca721-1af5-4de0-be16-5aaa5252ba04 admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:22:20.099116 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-997ca721-1af5-4de0-be16-5aaa5252ba04 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:22:20.117398 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997ca721-1af5-4de0-be16-5aaa5252ba04 admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:22:20.233756 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f444e3f-4e53-42d7-80e6-e02176b76c31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:22:20.234500 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f444e3f-4e53-42d7-80e6-e02176b76c31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:20.239483 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f444e3f-4e53-42d7-80e6-e02176b76c31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:20.239888 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f444e3f-4e53-42d7-80e6-e02176b76c31 admin admin] PATH_INFO: `/v3/projects/130c4dd2e5424c12b0906656c39bc9b6` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:20.241811 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f444e3f-4e53-42d7-80e6-e02176b76c31 admin admin] RBAC: Authorizing `identity:delete_project(project_id=130c4dd2e5424c12b0906656c39bc9b6)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:22:20.243840 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f444e3f-4e53-42d7-80e6-e02176b76c31 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:22:20.475298 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-8f444e3f-4e53-42d7-80e6-e02176b76c31 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '130c4dd2e5424c12b0906656c39bc9b6', 'request_id': 'req-8f444e3f-4e53-42d7-80e6-e02176b76c31'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:22:20.486467 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-8f444e3f-4e53-42d7-80e6-e02176b76c31 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '130c4dd2e5424c12b0906656c39bc9b6', 'request_id': 'req-8f444e3f-4e53-42d7-80e6-e02176b76c31'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:22:20.598327 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-8f444e3f-4e53-42d7-80e6-e02176b76c31 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '130c4dd2e5424c12b0906656c39bc9b6', 'request_id': 'req-8f444e3f-4e53-42d7-80e6-e02176b76c31'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:22:20.622141 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-8f444e3f-4e53-42d7-80e6-e02176b76c31 admin admin] The token cache is being invalidate because project 130c4dd2e5424c12b0906656c39bc9b6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81993) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 15:22:20.622499 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-8f444e3f-4e53-42d7-80e6-e02176b76c31 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:22:20.624232 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-8f444e3f-4e53-42d7-80e6-e02176b76c31 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:22:20.638244 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1077/2182] 10.222.0.55 () {58 vars in 1233 bytes} [Fri Sep 23 15:22:20 2022] DELETE /identity/v3/projects/130c4dd2e5424c12b0906656c39bc9b6 => generated 0 bytes in 531 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:22:20.638444 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f173e670-75f6-473e-9fed-c14f3cc4d0b8 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:20.638444 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f173e670-75f6-473e-9fed-c14f3cc4d0b8 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:20.638444 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f173e670-75f6-473e-9fed-c14f3cc4d0b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:20.664646 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-f173e670-75f6-473e-9fed-c14f3cc4d0b8 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:22:20.667017 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-f173e670-75f6-473e-9fed-c14f3cc4d0b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:22:20.765626 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1106/2183] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:22:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 15:22:20.774055 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25a5c63-bd2f-4ca8-a46b-c0448885b1ec None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:20.774650 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25a5c63-bd2f-4ca8-a46b-c0448885b1ec None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:20.775025 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25a5c63-bd2f-4ca8-a46b-c0448885b1ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:20.798065 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-d25a5c63-bd2f-4ca8-a46b-c0448885b1ec None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:22:20.799313 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-d25a5c63-bd2f-4ca8-a46b-c0448885b1ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:22:20.810684 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1078/2184] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:22:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 15:22:20.815496 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f173e670-75f6-473e-9fed-c14f3cc4d0b8 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:22:20.826482 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f173e670-75f6-473e-9fed-c14f3cc4d0b8 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:22:20.833272 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f551302a-4c5b-45bd-8997-8633e3ffc64e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:22:20.834024 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f551302a-4c5b-45bd-8997-8633e3ffc64e admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:20.834493 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f551302a-4c5b-45bd-8997-8633e3ffc64e admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:20.835000 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f551302a-4c5b-45bd-8997-8633e3ffc64e admin admin] PATH_INFO: `/v3/domains` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:20.837294 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f551302a-4c5b-45bd-8997-8633e3ffc64e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:22:20.837620 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f551302a-4c5b-45bd-8997-8633e3ffc64e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:22:20.839495 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f551302a-4c5b-45bd-8997-8633e3ffc64e admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:22:20.878629 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1107/2185] 10.222.0.55 () {58 vars in 1153 bytes} [Fri Sep 23 15:22:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:22:20.884664 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25a5c63-bd2f-4ca8-a46b-c0448885b1ec None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:22:20.896826 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25a5c63-bd2f-4ca8-a46b-c0448885b1ec None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:22:20.903611 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca76fcf3-eb4f-4243-b855-789b3bf497cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:22:20.904529 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca76fcf3-eb4f-4243-b855-789b3bf497cf admin admin] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:20.904816 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca76fcf3-eb4f-4243-b855-789b3bf497cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:20.905515 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca76fcf3-eb4f-4243-b855-789b3bf497cf admin admin] PATH_INFO: `/v3/projects` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:20.906176 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca76fcf3-eb4f-4243-b855-789b3bf497cf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:22:20.908833 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca76fcf3-eb4f-4243-b855-789b3bf497cf admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:22:21.041484 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1079/2186] 10.222.0.55 () {60 vars in 1153 bytes} [Fri Sep 23 15:22:20 2022] POST /identity/v3/projects => generated 381 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:22:21.046050 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f551302a-4c5b-45bd-8997-8633e3ffc64e admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:22:21.062246 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f551302a-4c5b-45bd-8997-8633e3ffc64e admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:22:21.164980 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4ab592-2375-415d-aa4b-c90c7038fa95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:22:21.165970 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4ab592-2375-415d-aa4b-c90c7038fa95 admin admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:21.166488 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4ab592-2375-415d-aa4b-c90c7038fa95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:21.167010 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4ab592-2375-415d-aa4b-c90c7038fa95 admin admin] PATH_INFO: `/v3/users` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:21.168070 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4ab592-2375-415d-aa4b-c90c7038fa95 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:22:21.171369 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4ab592-2375-415d-aa4b-c90c7038fa95 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:22:21.313072 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1108/2187] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Sep 23 15:22:21 2022] POST /identity/v3/users => generated 291 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:22:21.316640 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca76fcf3-eb4f-4243-b855-789b3bf497cf admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:22:21.330710 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca76fcf3-eb4f-4243-b855-789b3bf497cf admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:22:21.337371 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8750d20e-b5c9-415a-b2b3-7cc888834de5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:22:21.338392 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8750d20e-b5c9-415a-b2b3-7cc888834de5 admin admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:21.338939 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8750d20e-b5c9-415a-b2b3-7cc888834de5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:21.343782 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8750d20e-b5c9-415a-b2b3-7cc888834de5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:21.343782 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8750d20e-b5c9-415a-b2b3-7cc888834de5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:22:21.343782 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8750d20e-b5c9-415a-b2b3-7cc888834de5 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:22:21.358681 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1080/2188] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Sep 23 15:22:21 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:22:21.363455 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4ab592-2375-415d-aa4b-c90c7038fa95 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:22:21.377099 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4ab592-2375-415d-aa4b-c90c7038fa95 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:22:21.382094 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efe71eb-8177-4ce6-ac86-a0431134c4e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:22:21.382719 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efe71eb-8177-4ce6-ac86-a0431134c4e7 admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:21.382996 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efe71eb-8177-4ce6-ac86-a0431134c4e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:21.383270 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efe71eb-8177-4ce6-ac86-a0431134c4e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:21.384369 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efe71eb-8177-4ce6-ac86-a0431134c4e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:22:21.385980 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efe71eb-8177-4ce6-ac86-a0431134c4e7 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:22:21.403235 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1109/2189] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Sep 23 15:22:21 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:22:21.405829 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8750d20e-b5c9-415a-b2b3-7cc888834de5 admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:22:21.417444 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8750d20e-b5c9-415a-b2b3-7cc888834de5 admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:22:21.422554 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59af5972-b709-4471-8085-b9359eb17a79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:22:21.423447 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59af5972-b709-4471-8085-b9359eb17a79 admin admin] REQUEST_METHOD: `PUT` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:21.423764 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59af5972-b709-4471-8085-b9359eb17a79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:21.424039 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59af5972-b709-4471-8085-b9359eb17a79 admin admin] PATH_INFO: `/v3/projects/fea2d336f9e04a86aac184b9cd4c1734/users/162ab8b448fd4ea088c93f2c024a097a/roles/af257be2dee54d6b942f58650196582c` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:21.441046 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59af5972-b709-4471-8085-b9359eb17a79 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fea2d336f9e04a86aac184b9cd4c1734, user_id=162ab8b448fd4ea088c93f2c024a097a, role_id=af257be2dee54d6b942f58650196582c)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:22:21.443891 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59af5972-b709-4471-8085-b9359eb17a79 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:22:21.544798 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1081/2190] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Sep 23 15:22:21 2022] PUT /identity/v3/projects/fea2d336f9e04a86aac184b9cd4c1734/users/162ab8b448fd4ea088c93f2c024a097a/roles/af257be2dee54d6b942f58650196582c => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:22:21.560083 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9c6848-796a-4699-984d-939e1960f3fd None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:21.560421 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9c6848-796a-4699-984d-939e1960f3fd None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:21.560711 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9c6848-796a-4699-984d-939e1960f3fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:21.596788 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-bc9c6848-796a-4699-984d-939e1960f3fd None None] MFA Rules not processed for user `162ab8b448fd4ea088c93f2c024a097a`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:22:21.601230 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-bc9c6848-796a-4699-984d-939e1960f3fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:22:21.679448 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1110/2191] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:22:21 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:22:21.684050 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59af5972-b709-4471-8085-b9359eb17a79 admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:22:21.768077 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59af5972-b709-4471-8085-b9359eb17a79 admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:22:21.783833 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e737084d-0690-4488-a215-60fe98faa04f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:22:21.784902 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e737084d-0690-4488-a215-60fe98faa04f admin admin] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:21.785562 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e737084d-0690-4488-a215-60fe98faa04f admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:21.786008 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e737084d-0690-4488-a215-60fe98faa04f admin admin] PATH_INFO: `/v3/projects` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:21.786823 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e737084d-0690-4488-a215-60fe98faa04f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:22:21.789622 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e737084d-0690-4488-a215-60fe98faa04f admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:22:21.954752 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8cbf08-c3c5-4d47-bb5d-5eac41629b5f None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:21.955043 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8cbf08-c3c5-4d47-bb5d-5eac41629b5f None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:21.955323 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8cbf08-c3c5-4d47-bb5d-5eac41629b5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:21.978957 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-da8cbf08-c3c5-4d47-bb5d-5eac41629b5f None None] MFA Rules not processed for user `a5514abdc8b54282901367a4f2d02856`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:22:21.980476 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-da8cbf08-c3c5-4d47-bb5d-5eac41629b5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:22:22.029124 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1082/2192] 10.222.0.55 () {60 vars in 1153 bytes} [Fri Sep 23 15:22:21 2022] POST /identity/v3/projects => generated 381 bytes in 345 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:22:22.034644 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e737084d-0690-4488-a215-60fe98faa04f admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:22:22.119366 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1111/2193] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:22:21 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:22:22.214063 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e737084d-0690-4488-a215-60fe98faa04f admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:22:22.223333 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adec1872-4b94-4e02-88a6-01dc48feaa63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:22:22.224210 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adec1872-4b94-4e02-88a6-01dc48feaa63 admin admin] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:22.224495 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adec1872-4b94-4e02-88a6-01dc48feaa63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:22.224770 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adec1872-4b94-4e02-88a6-01dc48feaa63 admin admin] PATH_INFO: `/v3/users` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:22.225414 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adec1872-4b94-4e02-88a6-01dc48feaa63 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:22:22.227414 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adec1872-4b94-4e02-88a6-01dc48feaa63 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:22:22.322759 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1083/2194] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Sep 23 15:22:22 2022] POST /identity/v3/users => generated 297 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:22:22.335106 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8cbf08-c3c5-4d47-bb5d-5eac41629b5f None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:22:22.346122 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8cbf08-c3c5-4d47-bb5d-5eac41629b5f None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:22:22.351851 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b0ce1d-452b-4acf-9dc6-f4e86553a67b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:22:22.352728 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b0ce1d-452b-4acf-9dc6-f4e86553a67b admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:22.353151 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b0ce1d-452b-4acf-9dc6-f4e86553a67b admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:22.353576 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b0ce1d-452b-4acf-9dc6-f4e86553a67b admin admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:22.354752 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b0ce1d-452b-4acf-9dc6-f4e86553a67b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:22:22.356794 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b0ce1d-452b-4acf-9dc6-f4e86553a67b admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:22:22.388386 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1112/2195] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Sep 23 15:22:22 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:22:22.393827 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adec1872-4b94-4e02-88a6-01dc48feaa63 admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:22:22.414680 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adec1872-4b94-4e02-88a6-01dc48feaa63 admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:22:22.423597 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b406ca-2b86-419c-aa0e-e87ec2826f93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:22:22.424367 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b406ca-2b86-419c-aa0e-e87ec2826f93 admin admin] REQUEST_METHOD: `PUT` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:22.424663 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b406ca-2b86-419c-aa0e-e87ec2826f93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:22.424938 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b406ca-2b86-419c-aa0e-e87ec2826f93 admin admin] PATH_INFO: `/v3/projects/3e16afd6f05f485c9b104438d2815337/users/d454b1ef82fb46f5b92c09614471a887/roles/8e2919e79df040e8926f21607818b58c` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:22.461076 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b406ca-2b86-419c-aa0e-e87ec2826f93 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e16afd6f05f485c9b104438d2815337, user_id=d454b1ef82fb46f5b92c09614471a887, role_id=8e2919e79df040e8926f21607818b58c)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:22:22.465072 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b406ca-2b86-419c-aa0e-e87ec2826f93 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:22:22.624442 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1084/2196] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Sep 23 15:22:22 2022] PUT /identity/v3/projects/3e16afd6f05f485c9b104438d2815337/users/d454b1ef82fb46f5b92c09614471a887/roles/8e2919e79df040e8926f21607818b58c => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:22:22.629444 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b0ce1d-452b-4acf-9dc6-f4e86553a67b admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:22:22.652948 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b0ce1d-452b-4acf-9dc6-f4e86553a67b admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:22:22.776293 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094ed62f-9b09-4c53-9ba1-b2651e95c2fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:22:22.777200 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094ed62f-9b09-4c53-9ba1-b2651e95c2fb admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:22.777738 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094ed62f-9b09-4c53-9ba1-b2651e95c2fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:22.778632 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094ed62f-9b09-4c53-9ba1-b2651e95c2fb admin admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:22.780777 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094ed62f-9b09-4c53-9ba1-b2651e95c2fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:22:22.782947 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094ed62f-9b09-4c53-9ba1-b2651e95c2fb admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:22:22.798064 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1113/2197] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Sep 23 15:22:22 2022] GET /identity/v3/roles => generated 1420 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:22:22.802986 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b406ca-2b86-419c-aa0e-e87ec2826f93 admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:22:22.819951 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b406ca-2b86-419c-aa0e-e87ec2826f93 admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:22:22.878125 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b14aac6-8ba3-4a47-869a-fb3c1848d008 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:22:22.878125 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b14aac6-8ba3-4a47-869a-fb3c1848d008 admin admin] REQUEST_METHOD: `PUT` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:22.878125 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b14aac6-8ba3-4a47-869a-fb3c1848d008 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:22.878125 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b14aac6-8ba3-4a47-869a-fb3c1848d008 admin admin] PATH_INFO: `/v3/projects/3e16afd6f05f485c9b104438d2815337/users/d454b1ef82fb46f5b92c09614471a887/roles/8e2919e79df040e8926f21607818b58c` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:22.879951 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b14aac6-8ba3-4a47-869a-fb3c1848d008 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e16afd6f05f485c9b104438d2815337, user_id=d454b1ef82fb46f5b92c09614471a887, role_id=8e2919e79df040e8926f21607818b58c)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:22:22.879951 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b14aac6-8ba3-4a47-869a-fb3c1848d008 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:22:22.879951 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1085/2198] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Sep 23 15:22:22 2022] PUT /identity/v3/projects/3e16afd6f05f485c9b104438d2815337/users/d454b1ef82fb46f5b92c09614471a887/roles/8e2919e79df040e8926f21607818b58c => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:22:22.921114 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cb5e58-322d-4aef-b933-ff6daead8777 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:22.921583 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cb5e58-322d-4aef-b933-ff6daead8777 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:22.922652 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cb5e58-322d-4aef-b933-ff6daead8777 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:22.958211 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-28cb5e58-322d-4aef-b933-ff6daead8777 None None] MFA Rules not processed for user `d454b1ef82fb46f5b92c09614471a887`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:22:22.959462 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-28cb5e58-322d-4aef-b933-ff6daead8777 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:22:23.051654 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1114/2199] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:22:22 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:22:23.060407 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b14aac6-8ba3-4a47-869a-fb3c1848d008 admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:22:23.062433 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-4b14aac6-8ba3-4a47-869a-fb3c1848d008 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:22:23.141708 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b14aac6-8ba3-4a47-869a-fb3c1848d008 admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:22:23.145764 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b153f851-bace-4e38-82a0-c4a4a2e09eda service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feca6278be904ef8b0dc580a67e477f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:22:23.146399 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b153f851-bace-4e38-82a0-c4a4a2e09eda service cinder] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:23.146658 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b153f851-bace-4e38-82a0-c4a4a2e09eda service cinder] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:23.146928 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b153f851-bace-4e38-82a0-c4a4a2e09eda service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:23.173904 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b153f851-bace-4e38-82a0-c4a4a2e09eda service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:22:23.173904 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b153f851-bace-4e38-82a0-c4a4a2e09eda service cinder] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:22:23.188141 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1086/2200] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:22:23 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:22:23.315099 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cb5e58-322d-4aef-b933-ff6daead8777 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:22:23.329006 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cb5e58-322d-4aef-b933-ff6daead8777 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:22:23.334531 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e7dbf4-f29c-479c-baec-313894efa013 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feca6278be904ef8b0dc580a67e477f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:22:23.335215 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e7dbf4-f29c-479c-baec-313894efa013 service cinder] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:23.335500 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e7dbf4-f29c-479c-baec-313894efa013 service cinder] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:23.335773 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e7dbf4-f29c-479c-baec-313894efa013 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:23.430101 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e7dbf4-f29c-479c-baec-313894efa013 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:22:23.431749 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e7dbf4-f29c-479c-baec-313894efa013 service cinder] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:22:23.443453 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1115/2201] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:22:23 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:22:23.759671 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:22:24.951344 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:22:25.916750 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09a4167-0efe-47ac-a8a5-c2e2aff6f9e0 None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:25.917034 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09a4167-0efe-47ac-a8a5-c2e2aff6f9e0 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:25.917304 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09a4167-0efe-47ac-a8a5-c2e2aff6f9e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:25.952337 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-a09a4167-0efe-47ac-a8a5-c2e2aff6f9e0 None None] MFA Rules not processed for user `3a349a48d97d4d0283d630c55b36bd18`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:22:25.954165 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-a09a4167-0efe-47ac-a8a5-c2e2aff6f9e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:22:26.036709 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1087/2202] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:22:25 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:22:26.055772 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e7dbf4-f29c-479c-baec-313894efa013 service cinder] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:22:26.057281 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-29e7dbf4-f29c-479c-baec-313894efa013 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:22:26.131811 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e7dbf4-f29c-479c-baec-313894efa013 service cinder] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:22:26.137686 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814ae39c-f8a1-43a9-a307-146bb45d08c7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3c6c7a1325a47be924268e1aa1a7a18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:22:26.138314 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814ae39c-f8a1-43a9-a307-146bb45d08c7 service nova] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:26.138590 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814ae39c-f8a1-43a9-a307-146bb45d08c7 service nova] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:26.138865 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814ae39c-f8a1-43a9-a307-146bb45d08c7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:26.141580 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-814ae39c-f8a1-43a9-a307-146bb45d08c7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:22:26.198583 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814ae39c-f8a1-43a9-a307-146bb45d08c7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:22:26.200211 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814ae39c-f8a1-43a9-a307-146bb45d08c7 service nova] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:22:26.209909 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1116/2203] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:22:26 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:22:26.236972 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:22:26.763249 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:22:27.979883 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:22:28.562445 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:22:29.600767 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:22:30.601010 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09a4167-0efe-47ac-a8a5-c2e2aff6f9e0 None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:22:30.688250 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09a4167-0efe-47ac-a8a5-c2e2aff6f9e0 None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:22:30.693556 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb4a867-0da4-49f7-abe4-ae48288adbda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:22:30.694319 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb4a867-0da4-49f7-abe4-ae48288adbda admin admin] REQUEST_METHOD: `DELETE` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:30.694688 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb4a867-0da4-49f7-abe4-ae48288adbda admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:30.695025 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb4a867-0da4-49f7-abe4-ae48288adbda admin admin] PATH_INFO: `/v3/users/162ab8b448fd4ea088c93f2c024a097a` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:30.696385 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb4a867-0da4-49f7-abe4-ae48288adbda admin admin] RBAC: Authorizing `identity:delete_user(user_id=162ab8b448fd4ea088c93f2c024a097a)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:22:30.698888 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb4a867-0da4-49f7-abe4-ae48288adbda admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:22:30.996705 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ace653d-0fb7-498f-9d6f-3c0447c9ac0b None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:30.997576 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ace653d-0fb7-498f-9d6f-3c0447c9ac0b None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:30.998000 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ace653d-0fb7-498f-9d6f-3c0447c9ac0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:31.030282 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-7ace653d-0fb7-498f-9d6f-3c0447c9ac0b None None] MFA Rules not processed for user `4bdaecd5dd5d461383878c4b96ef98be`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:22:31.030282 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-7ace653d-0fb7-498f-9d6f-3c0447c9ac0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:22:31.041161 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1117/2204] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:22:30 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 15:22:31.096793 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-7bb4a867-0da4-49f7-abe4-ae48288adbda admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '162ab8b448fd4ea088c93f2c024a097a'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:22:31.133882 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-7bb4a867-0da4-49f7-abe4-ae48288adbda admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '162ab8b448fd4ea088c93f2c024a097a'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:22:31.143683 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-7bb4a867-0da4-49f7-abe4-ae48288adbda admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '162ab8b448fd4ea088c93f2c024a097a'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:22:31.310378 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-7bb4a867-0da4-49f7-abe4-ae48288adbda admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '162ab8b448fd4ea088c93f2c024a097a'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:22:31.311150 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-7bb4a867-0da4-49f7-abe4-ae48288adbda admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '162ab8b448fd4ea088c93f2c024a097a'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:22:31.320016 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1088/2205] 10.222.0.55 () {58 vars in 1224 bytes} [Fri Sep 23 15:22:30 2022] DELETE /identity/v3/users/162ab8b448fd4ea088c93f2c024a097a => generated 0 bytes in 719 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:22:31.324818 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2779fe-1524-42d7-b7e6-8491ed691eb8 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:31.325105 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2779fe-1524-42d7-b7e6-8491ed691eb8 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:31.325377 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2779fe-1524-42d7-b7e6-8491ed691eb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:31.346331 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-7d2779fe-1524-42d7-b7e6-8491ed691eb8 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:22:31.347771 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-7d2779fe-1524-42d7-b7e6-8491ed691eb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:22:31.423270 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1118/2206] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:22:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:22:31.433223 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb4a867-0da4-49f7-abe4-ae48288adbda admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:22:31.434845 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-7bb4a867-0da4-49f7-abe4-ae48288adbda admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:22:31.495961 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb4a867-0da4-49f7-abe4-ae48288adbda admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:22:31.500538 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ade9ee-c15e-40f5-accc-baf3769a21d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd259178570864ffeb936ee10dfec6b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:22:31.501293 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ade9ee-c15e-40f5-accc-baf3769a21d7 service neutron] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:31.501571 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ade9ee-c15e-40f5-accc-baf3769a21d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:31.501837 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ade9ee-c15e-40f5-accc-baf3769a21d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:31.519285 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ade9ee-c15e-40f5-accc-baf3769a21d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:22:31.519285 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ade9ee-c15e-40f5-accc-baf3769a21d7 service neutron] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:22:31.529988 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1089/2207] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:22:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:22:32.254521 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2779fe-1524-42d7-b7e6-8491ed691eb8 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:22:32.256583 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-7d2779fe-1524-42d7-b7e6-8491ed691eb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:22:32.285910 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2779fe-1524-42d7-b7e6-8491ed691eb8 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:22:32.295679 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:INSERT=2,keystone:SELECT=88,keystone:DELETE=10 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:22:32.315280 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31071769-cd13-4825-abb8-bace11be67a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:22:32.316217 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31071769-cd13-4825-abb8-bace11be67a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:32.316653 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31071769-cd13-4825-abb8-bace11be67a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:32.317362 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31071769-cd13-4825-abb8-bace11be67a3 admin admin] PATH_INFO: `/v3/projects/fea2d336f9e04a86aac184b9cd4c1734` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:32.319577 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31071769-cd13-4825-abb8-bace11be67a3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fea2d336f9e04a86aac184b9cd4c1734)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:22:32.330604 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31071769-cd13-4825-abb8-bace11be67a3 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:22:32.511703 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-31071769-cd13-4825-abb8-bace11be67a3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fea2d336f9e04a86aac184b9cd4c1734', 'request_id': 'req-31071769-cd13-4825-abb8-bace11be67a3'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:22:32.532879 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-31071769-cd13-4825-abb8-bace11be67a3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fea2d336f9e04a86aac184b9cd4c1734', 'request_id': 'req-31071769-cd13-4825-abb8-bace11be67a3'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:22:32.539146 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-31071769-cd13-4825-abb8-bace11be67a3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fea2d336f9e04a86aac184b9cd4c1734', 'request_id': 'req-31071769-cd13-4825-abb8-bace11be67a3'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:22:32.637031 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-31071769-cd13-4825-abb8-bace11be67a3 admin admin] The token cache is being invalidate because project fea2d336f9e04a86aac184b9cd4c1734 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81994) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 15:22:32.637337 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-31071769-cd13-4825-abb8-bace11be67a3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:22:32.638368 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-31071769-cd13-4825-abb8-bace11be67a3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:22:32.640170 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1119/2208] 10.222.0.55 () {58 vars in 1233 bytes} [Fri Sep 23 15:22:32 2022] DELETE /identity/v3/projects/fea2d336f9e04a86aac184b9cd4c1734 => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:22:32.645064 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ade9ee-c15e-40f5-accc-baf3769a21d7 service neutron] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:22:32.657712 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ade9ee-c15e-40f5-accc-baf3769a21d7 service neutron] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:22:32.722329 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786674e2-9ab7-440e-b80d-38fd1df09f67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:22:32.722976 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786674e2-9ab7-440e-b80d-38fd1df09f67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:32.723245 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786674e2-9ab7-440e-b80d-38fd1df09f67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:32.723564 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786674e2-9ab7-440e-b80d-38fd1df09f67 admin admin] PATH_INFO: `/v3/users/d454b1ef82fb46f5b92c09614471a887` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:32.724952 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786674e2-9ab7-440e-b80d-38fd1df09f67 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d454b1ef82fb46f5b92c09614471a887)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:22:32.727251 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786674e2-9ab7-440e-b80d-38fd1df09f67 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:22:33.115502 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-786674e2-9ab7-440e-b80d-38fd1df09f67 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd454b1ef82fb46f5b92c09614471a887'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:22:33.134275 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-786674e2-9ab7-440e-b80d-38fd1df09f67 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd454b1ef82fb46f5b92c09614471a887'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:22:33.144885 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-786674e2-9ab7-440e-b80d-38fd1df09f67 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd454b1ef82fb46f5b92c09614471a887'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:22:33.259499 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-786674e2-9ab7-440e-b80d-38fd1df09f67 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd454b1ef82fb46f5b92c09614471a887'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:22:33.260282 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-786674e2-9ab7-440e-b80d-38fd1df09f67 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd454b1ef82fb46f5b92c09614471a887'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:22:33.267595 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1090/2209] 10.222.0.55 () {58 vars in 1224 bytes} [Fri Sep 23 15:22:32 2022] DELETE /identity/v3/users/d454b1ef82fb46f5b92c09614471a887 => generated 0 bytes in 623 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:22:33.809715 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31071769-cd13-4825-abb8-bace11be67a3 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:22:33.811234 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-31071769-cd13-4825-abb8-bace11be67a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:22:33.825894 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31071769-cd13-4825-abb8-bace11be67a3 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:22:33.898473 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce36bd75-9629-482d-9559-13b640dee4bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:22:33.899157 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce36bd75-9629-482d-9559-13b640dee4bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:33.899443 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce36bd75-9629-482d-9559-13b640dee4bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:33.899721 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce36bd75-9629-482d-9559-13b640dee4bd admin admin] PATH_INFO: `/v3/projects/3e16afd6f05f485c9b104438d2815337` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:33.901008 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce36bd75-9629-482d-9559-13b640dee4bd admin admin] RBAC: Authorizing `identity:delete_project(project_id=3e16afd6f05f485c9b104438d2815337)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:22:33.903473 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce36bd75-9629-482d-9559-13b640dee4bd admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:22:34.191926 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-ce36bd75-9629-482d-9559-13b640dee4bd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3e16afd6f05f485c9b104438d2815337', 'request_id': 'req-ce36bd75-9629-482d-9559-13b640dee4bd'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:22:34.198186 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-ce36bd75-9629-482d-9559-13b640dee4bd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3e16afd6f05f485c9b104438d2815337', 'request_id': 'req-ce36bd75-9629-482d-9559-13b640dee4bd'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:22:34.204366 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-ce36bd75-9629-482d-9559-13b640dee4bd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3e16afd6f05f485c9b104438d2815337', 'request_id': 'req-ce36bd75-9629-482d-9559-13b640dee4bd'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:22:34.422148 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-ce36bd75-9629-482d-9559-13b640dee4bd admin admin] The token cache is being invalidate because project 3e16afd6f05f485c9b104438d2815337 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81994) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 15:22:34.422915 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-ce36bd75-9629-482d-9559-13b640dee4bd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:22:34.423890 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-ce36bd75-9629-482d-9559-13b640dee4bd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:22:34.426134 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:22:34.426624 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1120/2210] 10.222.0.55 () {58 vars in 1233 bytes} [Fri Sep 23 15:22:33 2022] DELETE /identity/v3/projects/3e16afd6f05f485c9b104438d2815337 => generated 0 bytes in 617 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:22:34.432945 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560699f0-3357-4932-9e7b-79fa8e52bf05 None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:34.437360 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560699f0-3357-4932-9e7b-79fa8e52bf05 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:34.437360 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560699f0-3357-4932-9e7b-79fa8e52bf05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:34.463481 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-560699f0-3357-4932-9e7b-79fa8e52bf05 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:22:34.465011 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-560699f0-3357-4932-9e7b-79fa8e52bf05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:22:34.533784 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1091/2211] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:22:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 15:22:34.543011 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7cb798-31de-4d66-93ca-148037da1c69 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:34.543298 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7cb798-31de-4d66-93ca-148037da1c69 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:34.543577 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7cb798-31de-4d66-93ca-148037da1c69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:34.563959 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-aa7cb798-31de-4d66-93ca-148037da1c69 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:22:34.565196 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-aa7cb798-31de-4d66-93ca-148037da1c69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:22:34.573041 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1121/2212] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:22:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:22:34.578028 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560699f0-3357-4932-9e7b-79fa8e52bf05 None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:22:34.587696 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560699f0-3357-4932-9e7b-79fa8e52bf05 None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:22:34.592682 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8794934a-0acc-46c4-8227-98d6affbf92e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:22:34.594482 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8794934a-0acc-46c4-8227-98d6affbf92e admin admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:34.594482 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8794934a-0acc-46c4-8227-98d6affbf92e admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:34.594711 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8794934a-0acc-46c4-8227-98d6affbf92e admin admin] PATH_INFO: `/v3/domains` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:34.595936 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8794934a-0acc-46c4-8227-98d6affbf92e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:22:34.596303 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8794934a-0acc-46c4-8227-98d6affbf92e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:22:34.598036 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8794934a-0acc-46c4-8227-98d6affbf92e admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:22:34.617768 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1092/2213] 10.222.0.55 () {58 vars in 1153 bytes} [Fri Sep 23 15:22:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:22:34.619113 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7cb798-31de-4d66-93ca-148037da1c69 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:22:34.630997 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7cb798-31de-4d66-93ca-148037da1c69 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:22:34.636711 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d744d6bb-4e5e-4a08-94eb-8f103fae7d0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:22:34.637787 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d744d6bb-4e5e-4a08-94eb-8f103fae7d0a admin admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:34.638100 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d744d6bb-4e5e-4a08-94eb-8f103fae7d0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:34.638387 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d744d6bb-4e5e-4a08-94eb-8f103fae7d0a admin admin] PATH_INFO: `/v3/projects` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:34.639162 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d744d6bb-4e5e-4a08-94eb-8f103fae7d0a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:22:34.643856 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d744d6bb-4e5e-4a08-94eb-8f103fae7d0a admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:22:34.845863 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1122/2214] 10.222.0.55 () {60 vars in 1153 bytes} [Fri Sep 23 15:22:34 2022] POST /identity/v3/projects => generated 379 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:22:34.850409 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8794934a-0acc-46c4-8227-98d6affbf92e admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:22:34.870019 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8794934a-0acc-46c4-8227-98d6affbf92e admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:22:34.939062 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ec6a92-b233-4b1d-9146-27a82930e059 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:22:34.940219 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ec6a92-b233-4b1d-9146-27a82930e059 admin admin] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:34.940509 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ec6a92-b233-4b1d-9146-27a82930e059 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:34.940790 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ec6a92-b233-4b1d-9146-27a82930e059 admin admin] PATH_INFO: `/v3/users` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:34.941586 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ec6a92-b233-4b1d-9146-27a82930e059 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:22:34.944068 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ec6a92-b233-4b1d-9146-27a82930e059 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:22:35.217049 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d744d6bb-4e5e-4a08-94eb-8f103fae7d0a admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:22:35.217739 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1093/2215] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Sep 23 15:22:34 2022] POST /identity/v3/users => generated 290 bytes in 367 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:22:35.233722 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d744d6bb-4e5e-4a08-94eb-8f103fae7d0a admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:22:35.241715 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbca4c79-1ced-415c-b699-4d188b97c09d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:22:35.242846 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbca4c79-1ced-415c-b699-4d188b97c09d admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:35.243071 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbca4c79-1ced-415c-b699-4d188b97c09d admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:35.243531 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbca4c79-1ced-415c-b699-4d188b97c09d admin admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:35.244290 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbca4c79-1ced-415c-b699-4d188b97c09d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:22:35.247071 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbca4c79-1ced-415c-b699-4d188b97c09d admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:22:35.261406 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1123/2216] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Sep 23 15:22:35 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:22:35.267890 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ec6a92-b233-4b1d-9146-27a82930e059 admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:22:35.283073 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ec6a92-b233-4b1d-9146-27a82930e059 admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:22:35.300295 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f483a53c-b393-4671-8cc5-6c9220d8a702 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:22:35.301288 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f483a53c-b393-4671-8cc5-6c9220d8a702 admin admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:35.301793 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f483a53c-b393-4671-8cc5-6c9220d8a702 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:35.302243 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f483a53c-b393-4671-8cc5-6c9220d8a702 admin admin] PATH_INFO: `/v3/roles` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:35.303069 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f483a53c-b393-4671-8cc5-6c9220d8a702 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:22:35.305729 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f483a53c-b393-4671-8cc5-6c9220d8a702 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:22:35.323996 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1094/2217] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Sep 23 15:22:35 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:22:35.326146 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbca4c79-1ced-415c-b699-4d188b97c09d admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:22:35.341946 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbca4c79-1ced-415c-b699-4d188b97c09d admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:22:35.346186 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b40603-2dc2-424f-8902-44f5320fed09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:22:35.348608 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b40603-2dc2-424f-8902-44f5320fed09 admin admin] REQUEST_METHOD: `PUT` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:35.348988 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b40603-2dc2-424f-8902-44f5320fed09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:35.349500 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b40603-2dc2-424f-8902-44f5320fed09 admin admin] PATH_INFO: `/v3/projects/1b5d86a1c1e44e27b0ecec583d78e331/users/b19e81a5e74348faacbffcea795bd13b/roles/af257be2dee54d6b942f58650196582c` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:35.366250 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b40603-2dc2-424f-8902-44f5320fed09 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1b5d86a1c1e44e27b0ecec583d78e331, user_id=b19e81a5e74348faacbffcea795bd13b, role_id=af257be2dee54d6b942f58650196582c)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:22:35.368934 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b40603-2dc2-424f-8902-44f5320fed09 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:22:35.527404 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1124/2218] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Sep 23 15:22:35 2022] PUT /identity/v3/projects/1b5d86a1c1e44e27b0ecec583d78e331/users/b19e81a5e74348faacbffcea795bd13b/roles/af257be2dee54d6b942f58650196582c => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:22:35.538252 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30988a4a-98ba-4236-86a0-d10ee508a5df None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:35.539093 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30988a4a-98ba-4236-86a0-d10ee508a5df None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:35.539375 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30988a4a-98ba-4236-86a0-d10ee508a5df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:35.575243 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-30988a4a-98ba-4236-86a0-d10ee508a5df None None] MFA Rules not processed for user `b19e81a5e74348faacbffcea795bd13b`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:22:35.576820 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-30988a4a-98ba-4236-86a0-d10ee508a5df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:22:35.679816 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1095/2219] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:22:35 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:22:35.684262 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b40603-2dc2-424f-8902-44f5320fed09 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:22:35.751009 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b40603-2dc2-424f-8902-44f5320fed09 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:22:35.755615 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27fbd7e-63c1-43d6-823d-180b88a73ee0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:22:35.756253 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27fbd7e-63c1-43d6-823d-180b88a73ee0 admin admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:35.756533 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27fbd7e-63c1-43d6-823d-180b88a73ee0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:35.756801 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27fbd7e-63c1-43d6-823d-180b88a73ee0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:35.757426 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27fbd7e-63c1-43d6-823d-180b88a73ee0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:22:35.759685 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27fbd7e-63c1-43d6-823d-180b88a73ee0 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:22:35.850895 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1125/2220] 10.222.0.55 () {60 vars in 1153 bytes} [Fri Sep 23 15:22:35 2022] POST /identity/v3/projects => generated 379 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:22:35.855216 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30988a4a-98ba-4236-86a0-d10ee508a5df None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:22:35.868113 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30988a4a-98ba-4236-86a0-d10ee508a5df None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:22:35.936883 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d200f38-ba88-46bb-999f-83c8fc8f240e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:22:35.937498 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d200f38-ba88-46bb-999f-83c8fc8f240e admin admin] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:35.937813 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d200f38-ba88-46bb-999f-83c8fc8f240e admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:35.938032 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d200f38-ba88-46bb-999f-83c8fc8f240e admin admin] PATH_INFO: `/v3/users` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:35.938665 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d200f38-ba88-46bb-999f-83c8fc8f240e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:22:35.940643 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d200f38-ba88-46bb-999f-83c8fc8f240e admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:22:36.043094 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1096/2221] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Sep 23 15:22:35 2022] POST /identity/v3/users => generated 296 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:22:36.047767 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27fbd7e-63c1-43d6-823d-180b88a73ee0 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:22:36.058081 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27fbd7e-63c1-43d6-823d-180b88a73ee0 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:22:36.063488 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0e26e4-c7d2-43c4-853b-92d729d2a095 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:22:36.064288 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0e26e4-c7d2-43c4-853b-92d729d2a095 admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:36.064573 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0e26e4-c7d2-43c4-853b-92d729d2a095 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:36.064865 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0e26e4-c7d2-43c4-853b-92d729d2a095 admin admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:36.065509 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0e26e4-c7d2-43c4-853b-92d729d2a095 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:22:36.067130 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0e26e4-c7d2-43c4-853b-92d729d2a095 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:22:36.080763 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1126/2222] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Sep 23 15:22:36 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:22:36.085725 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d200f38-ba88-46bb-999f-83c8fc8f240e admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:22:36.098247 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d200f38-ba88-46bb-999f-83c8fc8f240e admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:22:36.103305 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f790658-3c34-435a-bbdd-490d9e0cea43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:22:36.103912 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f790658-3c34-435a-bbdd-490d9e0cea43 admin admin] REQUEST_METHOD: `PUT` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:36.104375 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f790658-3c34-435a-bbdd-490d9e0cea43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:36.104656 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f790658-3c34-435a-bbdd-490d9e0cea43 admin admin] PATH_INFO: `/v3/projects/15d00dd42f9f46c5a963a5e542d78943/users/893a3318f30a45dcad90fd6688199884/roles/8e2919e79df040e8926f21607818b58c` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:36.125246 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f790658-3c34-435a-bbdd-490d9e0cea43 admin admin] RBAC: Authorizing `identity:create_grant(project_id=15d00dd42f9f46c5a963a5e542d78943, user_id=893a3318f30a45dcad90fd6688199884, role_id=8e2919e79df040e8926f21607818b58c)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:22:36.125246 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f790658-3c34-435a-bbdd-490d9e0cea43 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:22:36.230416 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1097/2223] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Sep 23 15:22:36 2022] PUT /identity/v3/projects/15d00dd42f9f46c5a963a5e542d78943/users/893a3318f30a45dcad90fd6688199884/roles/8e2919e79df040e8926f21607818b58c => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 15:22:36.234579 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0e26e4-c7d2-43c4-853b-92d729d2a095 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:22:36.244605 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0e26e4-c7d2-43c4-853b-92d729d2a095 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:22:36.310446 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772ab8f1-d53c-4681-8a3a-63634952e4f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:22:36.311062 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772ab8f1-d53c-4681-8a3a-63634952e4f4 admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:36.311346 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772ab8f1-d53c-4681-8a3a-63634952e4f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:36.311616 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772ab8f1-d53c-4681-8a3a-63634952e4f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:36.312300 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772ab8f1-d53c-4681-8a3a-63634952e4f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:22:36.313962 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772ab8f1-d53c-4681-8a3a-63634952e4f4 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:22:36.325305 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1127/2224] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Sep 23 15:22:36 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:22:36.329783 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f790658-3c34-435a-bbdd-490d9e0cea43 admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:22:36.341178 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f790658-3c34-435a-bbdd-490d9e0cea43 admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:22:36.347445 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7509076-e7a8-4a50-a98c-28bd98877a98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:22:36.348221 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7509076-e7a8-4a50-a98c-28bd98877a98 admin admin] REQUEST_METHOD: `PUT` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:36.348631 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7509076-e7a8-4a50-a98c-28bd98877a98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:36.348777 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7509076-e7a8-4a50-a98c-28bd98877a98 admin admin] PATH_INFO: `/v3/projects/15d00dd42f9f46c5a963a5e542d78943/users/893a3318f30a45dcad90fd6688199884/roles/8e2919e79df040e8926f21607818b58c` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:36.350739 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7509076-e7a8-4a50-a98c-28bd98877a98 admin admin] RBAC: Authorizing `identity:create_grant(project_id=15d00dd42f9f46c5a963a5e542d78943, user_id=893a3318f30a45dcad90fd6688199884, role_id=8e2919e79df040e8926f21607818b58c)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:22:36.353792 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7509076-e7a8-4a50-a98c-28bd98877a98 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:22:36.365831 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1098/2225] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Sep 23 15:22:36 2022] PUT /identity/v3/projects/15d00dd42f9f46c5a963a5e542d78943/users/893a3318f30a45dcad90fd6688199884/roles/8e2919e79df040e8926f21607818b58c => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:22:36.378952 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74919d7-34f3-4d9c-93e4-8aa78bbfcd33 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:36.379275 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74919d7-34f3-4d9c-93e4-8aa78bbfcd33 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:36.379493 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74919d7-34f3-4d9c-93e4-8aa78bbfcd33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:36.406845 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7509076-e7a8-4a50-a98c-28bd98877a98 admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:22:36.408777 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-a7509076-e7a8-4a50-a98c-28bd98877a98 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:22:36.409661 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-c74919d7-34f3-4d9c-93e4-8aa78bbfcd33 None None] MFA Rules not processed for user `893a3318f30a45dcad90fd6688199884`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:22:36.410953 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-c74919d7-34f3-4d9c-93e4-8aa78bbfcd33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:22:36.494361 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1128/2226] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:22:36 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:22:36.498794 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7509076-e7a8-4a50-a98c-28bd98877a98 admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:22:36.503789 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74919d7-34f3-4d9c-93e4-8aa78bbfcd33 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:22:36.504181 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03a5376-9faf-461f-b8ce-93b2b09399ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:22:36.505999 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03a5376-9faf-461f-b8ce-93b2b09399ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:36.505999 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03a5376-9faf-461f-b8ce-93b2b09399ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:36.505999 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03a5376-9faf-461f-b8ce-93b2b09399ef admin admin] PATH_INFO: `/v3/users/1c5f5561caaa4223ad43f95bba10e739` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:36.506287 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-c74919d7-34f3-4d9c-93e4-8aa78bbfcd33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:22:36.506488 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03a5376-9faf-461f-b8ce-93b2b09399ef admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c5f5561caaa4223ad43f95bba10e739)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:22:36.508708 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03a5376-9faf-461f-b8ce-93b2b09399ef admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:22:36.567393 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74919d7-34f3-4d9c-93e4-8aa78bbfcd33 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:22:36.571041 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4459cb-6753-439b-b07f-f3ee45db90eb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feca6278be904ef8b0dc580a67e477f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:22:36.571645 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4459cb-6753-439b-b07f-f3ee45db90eb service cinder] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:36.571921 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4459cb-6753-439b-b07f-f3ee45db90eb service cinder] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:36.572233 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4459cb-6753-439b-b07f-f3ee45db90eb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:36.581776 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4459cb-6753-439b-b07f-f3ee45db90eb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:22:36.583259 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4459cb-6753-439b-b07f-f3ee45db90eb service cinder] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:22:36.594090 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1129/2227] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:22:36 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 23 15:22:36.742020 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:22:36.777680 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-a03a5376-9faf-461f-b8ce-93b2b09399ef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c5f5561caaa4223ad43f95bba10e739'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:22:36.816751 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-a03a5376-9faf-461f-b8ce-93b2b09399ef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c5f5561caaa4223ad43f95bba10e739'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:22:36.836623 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-a03a5376-9faf-461f-b8ce-93b2b09399ef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c5f5561caaa4223ad43f95bba10e739'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:22:36.944188 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-a03a5376-9faf-461f-b8ce-93b2b09399ef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c5f5561caaa4223ad43f95bba10e739'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:22:36.944827 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-a03a5376-9faf-461f-b8ce-93b2b09399ef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c5f5561caaa4223ad43f95bba10e739'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:22:36.978987 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1099/2228] 10.222.0.55 () {58 vars in 1224 bytes} [Fri Sep 23 15:22:36 2022] DELETE /identity/v3/users/1c5f5561caaa4223ad43f95bba10e739 => generated 0 bytes in 574 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 15:22:36.983705 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cac50fa-0e60-4c44-a549-fa656b4a03cb None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:36.984244 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cac50fa-0e60-4c44-a549-fa656b4a03cb None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:36.984549 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cac50fa-0e60-4c44-a549-fa656b4a03cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:37.008914 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-4cac50fa-0e60-4c44-a549-fa656b4a03cb None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:22:37.010403 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-4cac50fa-0e60-4c44-a549-fa656b4a03cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:22:37.073514 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1130/2229] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:22:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:22:37.084335 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03a5376-9faf-461f-b8ce-93b2b09399ef admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:22:37.086295 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-a03a5376-9faf-461f-b8ce-93b2b09399ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:22:37.148308 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03a5376-9faf-461f-b8ce-93b2b09399ef admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:22:37.151910 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd6f611-5ed5-4d14-8859-562e9b3fcb94 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd259178570864ffeb936ee10dfec6b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:22:37.152551 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd6f611-5ed5-4d14-8859-562e9b3fcb94 service neutron] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:37.152832 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd6f611-5ed5-4d14-8859-562e9b3fcb94 service neutron] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:37.153105 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd6f611-5ed5-4d14-8859-562e9b3fcb94 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:37.166578 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd6f611-5ed5-4d14-8859-562e9b3fcb94 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:22:37.168219 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd6f611-5ed5-4d14-8859-562e9b3fcb94 service neutron] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:22:37.179136 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1100/2230] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:22:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:22:37.220319 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cac50fa-0e60-4c44-a549-fa656b4a03cb None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:22:37.222431 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-4cac50fa-0e60-4c44-a549-fa656b4a03cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:22:37.238651 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cac50fa-0e60-4c44-a549-fa656b4a03cb None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:22:37.243549 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8377807-0a8e-4f25-944c-60097202445a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:22:37.244226 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8377807-0a8e-4f25-944c-60097202445a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:37.244806 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8377807-0a8e-4f25-944c-60097202445a admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:37.244930 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8377807-0a8e-4f25-944c-60097202445a admin admin] PATH_INFO: `/v3/users/b19e81a5e74348faacbffcea795bd13b` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:37.246183 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8377807-0a8e-4f25-944c-60097202445a admin admin] RBAC: Authorizing `identity:delete_user(user_id=b19e81a5e74348faacbffcea795bd13b)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:22:37.248149 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8377807-0a8e-4f25-944c-60097202445a admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:22:37.547396 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-a8377807-0a8e-4f25-944c-60097202445a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b19e81a5e74348faacbffcea795bd13b'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:22:37.551450 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:22:37.561812 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-a8377807-0a8e-4f25-944c-60097202445a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b19e81a5e74348faacbffcea795bd13b'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:22:37.570059 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-a8377807-0a8e-4f25-944c-60097202445a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b19e81a5e74348faacbffcea795bd13b'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:22:37.692980 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-a8377807-0a8e-4f25-944c-60097202445a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b19e81a5e74348faacbffcea795bd13b'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:22:37.693748 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-a8377807-0a8e-4f25-944c-60097202445a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b19e81a5e74348faacbffcea795bd13b'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:22:37.701942 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd6f611-5ed5-4d14-8859-562e9b3fcb94 service neutron] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:22:37.705734 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-2cd6f611-5ed5-4d14-8859-562e9b3fcb94 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:22:37.708361 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1131/2231] 10.222.0.55 () {58 vars in 1224 bytes} [Fri Sep 23 15:22:37 2022] DELETE /identity/v3/users/b19e81a5e74348faacbffcea795bd13b => generated 0 bytes in 486 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:22:37.710772 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb50cfe-e18c-4c18-8da0-be0318f8058a None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:37.711244 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb50cfe-e18c-4c18-8da0-be0318f8058a None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:37.711675 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb50cfe-e18c-4c18-8da0-be0318f8058a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:37.736525 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-1cb50cfe-e18c-4c18-8da0-be0318f8058a None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:22:37.736732 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd6f611-5ed5-4d14-8859-562e9b3fcb94 service neutron] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:22:37.736821 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-1cb50cfe-e18c-4c18-8da0-be0318f8058a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:22:37.798578 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4038b5-c391-4e65-bcc7-0c45b3403b0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:22:37.799218 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4038b5-c391-4e65-bcc7-0c45b3403b0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:37.799501 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4038b5-c391-4e65-bcc7-0c45b3403b0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:37.799775 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4038b5-c391-4e65-bcc7-0c45b3403b0b admin admin] PATH_INFO: `/v3/projects/9e93e2850373495cb1abd9a04f30d142` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:37.800928 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4038b5-c391-4e65-bcc7-0c45b3403b0b admin admin] RBAC: Authorizing `identity:delete_project(project_id=9e93e2850373495cb1abd9a04f30d142)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:22:37.802854 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4038b5-c391-4e65-bcc7-0c45b3403b0b admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:22:37.806185 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1132/2232] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:22:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:22:37.817454 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb50cfe-e18c-4c18-8da0-be0318f8058a None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:22:37.818743 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-1cb50cfe-e18c-4c18-8da0-be0318f8058a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:22:37.884312 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb50cfe-e18c-4c18-8da0-be0318f8058a None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:22:37.888040 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e354f9-0e95-44d6-854a-01804f9c9793 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd259178570864ffeb936ee10dfec6b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:22:37.888656 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e354f9-0e95-44d6-854a-01804f9c9793 service neutron] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:37.888936 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e354f9-0e95-44d6-854a-01804f9c9793 service neutron] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:37.889210 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e354f9-0e95-44d6-854a-01804f9c9793 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:37.900116 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e354f9-0e95-44d6-854a-01804f9c9793 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:22:37.901574 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e354f9-0e95-44d6-854a-01804f9c9793 service neutron] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:22:37.912460 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1133/2233] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:22:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 23 15:22:37.955045 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-fc4038b5-c391-4e65-bcc7-0c45b3403b0b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9e93e2850373495cb1abd9a04f30d142', 'request_id': 'req-fc4038b5-c391-4e65-bcc7-0c45b3403b0b'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:22:37.958893 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-fc4038b5-c391-4e65-bcc7-0c45b3403b0b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9e93e2850373495cb1abd9a04f30d142', 'request_id': 'req-fc4038b5-c391-4e65-bcc7-0c45b3403b0b'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:22:38.109607 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-fc4038b5-c391-4e65-bcc7-0c45b3403b0b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9e93e2850373495cb1abd9a04f30d142', 'request_id': 'req-fc4038b5-c391-4e65-bcc7-0c45b3403b0b'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:22:38.123122 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-fc4038b5-c391-4e65-bcc7-0c45b3403b0b admin admin] The token cache is being invalidate because project 9e93e2850373495cb1abd9a04f30d142 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81993) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 15:22:38.123544 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-fc4038b5-c391-4e65-bcc7-0c45b3403b0b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:22:38.125281 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-fc4038b5-c391-4e65-bcc7-0c45b3403b0b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:22:38.130735 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1101/2234] 10.222.0.55 () {58 vars in 1233 bytes} [Fri Sep 23 15:22:37 2022] DELETE /identity/v3/projects/9e93e2850373495cb1abd9a04f30d142 => generated 0 bytes in 427 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:22:38.132943 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e490dac-1342-4de4-a838-dfe810107c45 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:38.133742 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e490dac-1342-4de4-a838-dfe810107c45 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:38.134175 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e490dac-1342-4de4-a838-dfe810107c45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:38.164321 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-3e490dac-1342-4de4-a838-dfe810107c45 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:22:38.165819 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-3e490dac-1342-4de4-a838-dfe810107c45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:22:38.243641 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1134/2235] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:22:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 15:22:38.249952 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1c059c-c7bf-49ca-b196-41c2bcaacf85 None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:38.250321 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1c059c-c7bf-49ca-b196-41c2bcaacf85 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:38.250675 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1c059c-c7bf-49ca-b196-41c2bcaacf85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:38.271175 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-8b1c059c-c7bf-49ca-b196-41c2bcaacf85 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:22:38.272936 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-8b1c059c-c7bf-49ca-b196-41c2bcaacf85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:22:38.282978 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1102/2236] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:22:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:22:38.292250 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e490dac-1342-4de4-a838-dfe810107c45 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:22:38.303077 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e490dac-1342-4de4-a838-dfe810107c45 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:22:38.307787 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f21b35-8258-417f-b874-8c403d156a1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:22:38.308433 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f21b35-8258-417f-b874-8c403d156a1e admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:38.308715 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f21b35-8258-417f-b874-8c403d156a1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:38.308995 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f21b35-8258-417f-b874-8c403d156a1e admin admin] PATH_INFO: `/v3/domains` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:38.309641 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f21b35-8258-417f-b874-8c403d156a1e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:22:38.309948 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f21b35-8258-417f-b874-8c403d156a1e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:22:38.311662 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f21b35-8258-417f-b874-8c403d156a1e admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:22:38.335826 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1135/2237] 10.222.0.55 () {58 vars in 1153 bytes} [Fri Sep 23 15:22:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:22:38.342407 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1c059c-c7bf-49ca-b196-41c2bcaacf85 None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:22:38.353793 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1c059c-c7bf-49ca-b196-41c2bcaacf85 None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:22:38.360167 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a208e5a4-b46a-46d6-9ab0-b628ab38be2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:22:38.360780 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a208e5a4-b46a-46d6-9ab0-b628ab38be2c admin admin] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:38.361058 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a208e5a4-b46a-46d6-9ab0-b628ab38be2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:38.361323 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a208e5a4-b46a-46d6-9ab0-b628ab38be2c admin admin] PATH_INFO: `/v3/projects` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:38.361949 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a208e5a4-b46a-46d6-9ab0-b628ab38be2c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:22:38.364146 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a208e5a4-b46a-46d6-9ab0-b628ab38be2c admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:22:38.494307 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f21b35-8258-417f-b874-8c403d156a1e admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:22:38.496081 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-04f21b35-8258-417f-b874-8c403d156a1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:22:38.509510 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f21b35-8258-417f-b874-8c403d156a1e admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:22:38.515419 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973968e7-a668-4889-94b1-ccdc493529b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:22:38.516587 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973968e7-a668-4889-94b1-ccdc493529b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:38.516984 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973968e7-a668-4889-94b1-ccdc493529b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:38.517295 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973968e7-a668-4889-94b1-ccdc493529b0 admin admin] PATH_INFO: `/v3/projects/1b5d86a1c1e44e27b0ecec583d78e331` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:38.518622 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973968e7-a668-4889-94b1-ccdc493529b0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1b5d86a1c1e44e27b0ecec583d78e331)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:22:38.520584 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973968e7-a668-4889-94b1-ccdc493529b0 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:22:38.537341 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1103/2238] 10.222.0.55 () {60 vars in 1153 bytes} [Fri Sep 23 15:22:38 2022] POST /identity/v3/projects => generated 381 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:22:38.541551 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a208e5a4-b46a-46d6-9ab0-b628ab38be2c admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:22:38.604808 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a208e5a4-b46a-46d6-9ab0-b628ab38be2c admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:22:38.609497 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5cb685-2f6a-43e1-b6e8-d2710542851f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:22:38.610096 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5cb685-2f6a-43e1-b6e8-d2710542851f admin admin] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:38.610382 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5cb685-2f6a-43e1-b6e8-d2710542851f admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:38.610686 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5cb685-2f6a-43e1-b6e8-d2710542851f admin admin] PATH_INFO: `/v3/users` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:38.611305 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5cb685-2f6a-43e1-b6e8-d2710542851f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:22:38.613249 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5cb685-2f6a-43e1-b6e8-d2710542851f admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:22:38.665622 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-973968e7-a668-4889-94b1-ccdc493529b0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1b5d86a1c1e44e27b0ecec583d78e331', 'request_id': 'req-973968e7-a668-4889-94b1-ccdc493529b0'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:22:38.672910 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-973968e7-a668-4889-94b1-ccdc493529b0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1b5d86a1c1e44e27b0ecec583d78e331', 'request_id': 'req-973968e7-a668-4889-94b1-ccdc493529b0'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:22:38.678751 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-973968e7-a668-4889-94b1-ccdc493529b0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1b5d86a1c1e44e27b0ecec583d78e331', 'request_id': 'req-973968e7-a668-4889-94b1-ccdc493529b0'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:22:38.744626 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1104/2239] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Sep 23 15:22:38 2022] POST /identity/v3/users => generated 291 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 23 15:22:38.748677 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5cb685-2f6a-43e1-b6e8-d2710542851f admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:22:38.759766 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5cb685-2f6a-43e1-b6e8-d2710542851f admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:22:38.773769 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-973968e7-a668-4889-94b1-ccdc493529b0 admin admin] The token cache is being invalidate because project 1b5d86a1c1e44e27b0ecec583d78e331 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81994) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 15:22:38.782466 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-973968e7-a668-4889-94b1-ccdc493529b0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:22:38.782466 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-973968e7-a668-4889-94b1-ccdc493529b0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:22:38.782466 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1136/2240] 10.222.0.55 () {58 vars in 1233 bytes} [Fri Sep 23 15:22:38 2022] DELETE /identity/v3/projects/1b5d86a1c1e44e27b0ecec583d78e331 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 15:22:38.784074 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973968e7-a668-4889-94b1-ccdc493529b0 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:22:38.785480 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-973968e7-a668-4889-94b1-ccdc493529b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:22:38.834464 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f2e9f8-9e5d-481a-9a80-b36a85d55519 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:22:38.835066 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f2e9f8-9e5d-481a-9a80-b36a85d55519 admin admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:38.835344 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f2e9f8-9e5d-481a-9a80-b36a85d55519 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:38.835699 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f2e9f8-9e5d-481a-9a80-b36a85d55519 admin admin] PATH_INFO: `/v3/roles` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:38.836379 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f2e9f8-9e5d-481a-9a80-b36a85d55519 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:22:38.838086 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f2e9f8-9e5d-481a-9a80-b36a85d55519 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:22:38.851708 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1105/2241] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Sep 23 15:22:38 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:22:38.852755 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f2e9f8-9e5d-481a-9a80-b36a85d55519 admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:22:38.854515 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973968e7-a668-4889-94b1-ccdc493529b0 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:22:38.854659 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-41f2e9f8-9e5d-481a-9a80-b36a85d55519 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:22:38.859671 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547d62d2-bcbb-4262-89dd-6a7c423d9846 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:22:38.860342 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547d62d2-bcbb-4262-89dd-6a7c423d9846 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:38.860681 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547d62d2-bcbb-4262-89dd-6a7c423d9846 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:38.860967 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547d62d2-bcbb-4262-89dd-6a7c423d9846 admin admin] PATH_INFO: `/v3/users/893a3318f30a45dcad90fd6688199884` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:38.862151 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547d62d2-bcbb-4262-89dd-6a7c423d9846 admin admin] RBAC: Authorizing `identity:delete_user(user_id=893a3318f30a45dcad90fd6688199884)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:22:38.864316 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547d62d2-bcbb-4262-89dd-6a7c423d9846 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:22:38.868025 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f2e9f8-9e5d-481a-9a80-b36a85d55519 admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:22:38.873232 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceea56bd-0449-4d00-94e7-6a7a5bb57d2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:22:38.873941 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceea56bd-0449-4d00-94e7-6a7a5bb57d2c admin admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:38.874796 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceea56bd-0449-4d00-94e7-6a7a5bb57d2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:38.875078 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceea56bd-0449-4d00-94e7-6a7a5bb57d2c admin admin] PATH_INFO: `/v3/roles` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:38.875720 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceea56bd-0449-4d00-94e7-6a7a5bb57d2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:22:38.877437 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceea56bd-0449-4d00-94e7-6a7a5bb57d2c admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:22:38.889252 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1106/2242] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Sep 23 15:22:38 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:22:38.894216 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceea56bd-0449-4d00-94e7-6a7a5bb57d2c admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:22:38.908320 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceea56bd-0449-4d00-94e7-6a7a5bb57d2c admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:22:38.909965 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8878d2d2-13ff-4d04-8133-c8ccf823e7d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:22:38.910619 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8878d2d2-13ff-4d04-8133-c8ccf823e7d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:38.910898 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8878d2d2-13ff-4d04-8133-c8ccf823e7d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:38.911173 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8878d2d2-13ff-4d04-8133-c8ccf823e7d3 admin admin] PATH_INFO: `/v3/projects/c9488f4d4af441168341205e0e936ce4/users/ee65edb3211b4b71a771594a90a342ff/roles/af257be2dee54d6b942f58650196582c` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:38.925554 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8878d2d2-13ff-4d04-8133-c8ccf823e7d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c9488f4d4af441168341205e0e936ce4, user_id=ee65edb3211b4b71a771594a90a342ff, role_id=af257be2dee54d6b942f58650196582c)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:22:38.928193 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8878d2d2-13ff-4d04-8133-c8ccf823e7d3 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:22:39.090247 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1107/2243] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Sep 23 15:22:38 2022] PUT /identity/v3/projects/c9488f4d4af441168341205e0e936ce4/users/ee65edb3211b4b71a771594a90a342ff/roles/af257be2dee54d6b942f58650196582c => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 15:22:39.105098 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b467fc-aaac-4ea9-aa87-ba31403a68ea None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:39.105098 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b467fc-aaac-4ea9-aa87-ba31403a68ea None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:39.105098 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b467fc-aaac-4ea9-aa87-ba31403a68ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:39.116286 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-05b467fc-aaac-4ea9-aa87-ba31403a68ea None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:22:39.117706 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-05b467fc-aaac-4ea9-aa87-ba31403a68ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:22:39.138798 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-547d62d2-bcbb-4262-89dd-6a7c423d9846 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '893a3318f30a45dcad90fd6688199884'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:22:39.153550 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-547d62d2-bcbb-4262-89dd-6a7c423d9846 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '893a3318f30a45dcad90fd6688199884'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:22:39.161389 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-547d62d2-bcbb-4262-89dd-6a7c423d9846 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '893a3318f30a45dcad90fd6688199884'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:22:39.187437 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1108/2244] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:22:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 15:22:39.195358 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b467fc-aaac-4ea9-aa87-ba31403a68ea None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:22:39.201186 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-05b467fc-aaac-4ea9-aa87-ba31403a68ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:22:39.276456 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b467fc-aaac-4ea9-aa87-ba31403a68ea None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:22:39.280703 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d284711c-2fdb-4583-b28f-4ec0c7ebe34b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd259178570864ffeb936ee10dfec6b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:22:39.281489 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d284711c-2fdb-4583-b28f-4ec0c7ebe34b service neutron] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:39.281738 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-547d62d2-bcbb-4262-89dd-6a7c423d9846 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '893a3318f30a45dcad90fd6688199884'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:22:39.281855 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d284711c-2fdb-4583-b28f-4ec0c7ebe34b service neutron] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:39.282032 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d284711c-2fdb-4583-b28f-4ec0c7ebe34b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:39.282825 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-547d62d2-bcbb-4262-89dd-6a7c423d9846 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '893a3318f30a45dcad90fd6688199884'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:22:39.284159 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-d284711c-2fdb-4583-b28f-4ec0c7ebe34b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:22:39.290139 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1137/2245] 10.222.0.55 () {58 vars in 1224 bytes} [Fri Sep 23 15:22:38 2022] DELETE /identity/v3/users/893a3318f30a45dcad90fd6688199884 => generated 0 bytes in 507 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:22:39.300170 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d284711c-2fdb-4583-b28f-4ec0c7ebe34b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:22:39.301449 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d284711c-2fdb-4583-b28f-4ec0c7ebe34b service neutron] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:22:39.313101 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1109/2246] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:22:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 23 15:22:39.963018 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547d62d2-bcbb-4262-89dd-6a7c423d9846 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:22:39.964743 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-547d62d2-bcbb-4262-89dd-6a7c423d9846 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:22:40.055580 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547d62d2-bcbb-4262-89dd-6a7c423d9846 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:22:40.062710 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680434b8-c850-435b-a538-803ef4363fc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:22:40.069383 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680434b8-c850-435b-a538-803ef4363fc7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:40.069663 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680434b8-c850-435b-a538-803ef4363fc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:40.069946 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680434b8-c850-435b-a538-803ef4363fc7 admin admin] PATH_INFO: `/v3/projects/15d00dd42f9f46c5a963a5e542d78943` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:40.071410 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680434b8-c850-435b-a538-803ef4363fc7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=15d00dd42f9f46c5a963a5e542d78943)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:22:40.074064 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680434b8-c850-435b-a538-803ef4363fc7 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:22:40.263815 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-680434b8-c850-435b-a538-803ef4363fc7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '15d00dd42f9f46c5a963a5e542d78943', 'request_id': 'req-680434b8-c850-435b-a538-803ef4363fc7'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:22:40.271504 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-680434b8-c850-435b-a538-803ef4363fc7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '15d00dd42f9f46c5a963a5e542d78943', 'request_id': 'req-680434b8-c850-435b-a538-803ef4363fc7'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:22:40.277265 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-680434b8-c850-435b-a538-803ef4363fc7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '15d00dd42f9f46c5a963a5e542d78943', 'request_id': 'req-680434b8-c850-435b-a538-803ef4363fc7'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:22:40.378735 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-680434b8-c850-435b-a538-803ef4363fc7 admin admin] The token cache is being invalidate because project 15d00dd42f9f46c5a963a5e542d78943 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81994) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 15:22:40.379120 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-680434b8-c850-435b-a538-803ef4363fc7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:22:40.400618 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-680434b8-c850-435b-a538-803ef4363fc7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:22:40.400618 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1138/2247] 10.222.0.55 () {58 vars in 1233 bytes} [Fri Sep 23 15:22:39 2022] DELETE /identity/v3/projects/15d00dd42f9f46c5a963a5e542d78943 => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:22:40.406496 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce260df-7694-41af-a634-249eca86ccfa None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:40.406496 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce260df-7694-41af-a634-249eca86ccfa None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:40.406496 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce260df-7694-41af-a634-249eca86ccfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:40.445369 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-8ce260df-7694-41af-a634-249eca86ccfa None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:22:40.446773 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-8ce260df-7694-41af-a634-249eca86ccfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:22:40.536526 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1110/2248] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:22:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:22:40.544873 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e741bc-f3b6-4885-8889-e1818fb6a076 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:40.545160 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e741bc-f3b6-4885-8889-e1818fb6a076 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:40.545436 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e741bc-f3b6-4885-8889-e1818fb6a076 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:40.571038 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-c8e741bc-f3b6-4885-8889-e1818fb6a076 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:22:40.572283 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-c8e741bc-f3b6-4885-8889-e1818fb6a076 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:22:40.581926 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1139/2249] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:22:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:22:40.586397 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce260df-7694-41af-a634-249eca86ccfa None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:22:40.598091 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce260df-7694-41af-a634-249eca86ccfa None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:22:40.603482 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a195d8cd-e8b4-4763-a1ba-990e5e8c7976 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:22:40.604125 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a195d8cd-e8b4-4763-a1ba-990e5e8c7976 admin admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:40.604459 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a195d8cd-e8b4-4763-a1ba-990e5e8c7976 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:40.604734 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a195d8cd-e8b4-4763-a1ba-990e5e8c7976 admin admin] PATH_INFO: `/v3/domains` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:40.605383 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a195d8cd-e8b4-4763-a1ba-990e5e8c7976 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:22:40.605766 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a195d8cd-e8b4-4763-a1ba-990e5e8c7976 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:22:40.612034 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a195d8cd-e8b4-4763-a1ba-990e5e8c7976 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:22:40.638453 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1111/2250] 10.222.0.55 () {58 vars in 1153 bytes} [Fri Sep 23 15:22:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:22:40.642412 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:22:40.642615 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e741bc-f3b6-4885-8889-e1818fb6a076 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:22:40.654038 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e741bc-f3b6-4885-8889-e1818fb6a076 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:22:40.659479 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f63689d-dbf0-4a88-b7e6-ddd0e7e3130b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:22:40.660297 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f63689d-dbf0-4a88-b7e6-ddd0e7e3130b admin admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:40.660582 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f63689d-dbf0-4a88-b7e6-ddd0e7e3130b admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:40.660857 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f63689d-dbf0-4a88-b7e6-ddd0e7e3130b admin admin] PATH_INFO: `/v3/projects` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:40.661500 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f63689d-dbf0-4a88-b7e6-ddd0e7e3130b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:22:40.663563 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f63689d-dbf0-4a88-b7e6-ddd0e7e3130b admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:22:40.814196 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1140/2251] 10.222.0.55 () {60 vars in 1153 bytes} [Fri Sep 23 15:22:40 2022] POST /identity/v3/projects => generated 377 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:22:40.820357 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a195d8cd-e8b4-4763-a1ba-990e5e8c7976 admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:22:40.831574 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a195d8cd-e8b4-4763-a1ba-990e5e8c7976 admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:22:40.900537 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92976cc9-f98c-4da2-9859-715c64b7640e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:22:40.901249 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92976cc9-f98c-4da2-9859-715c64b7640e admin admin] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:40.901427 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92976cc9-f98c-4da2-9859-715c64b7640e admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:40.901691 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92976cc9-f98c-4da2-9859-715c64b7640e admin admin] PATH_INFO: `/v3/users` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:40.902491 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92976cc9-f98c-4da2-9859-715c64b7640e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:22:40.904806 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92976cc9-f98c-4da2-9859-715c64b7640e admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:22:41.066857 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1112/2252] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Sep 23 15:22:40 2022] POST /identity/v3/users => generated 289 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:22:41.071151 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f63689d-dbf0-4a88-b7e6-ddd0e7e3130b admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:22:41.086720 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f63689d-dbf0-4a88-b7e6-ddd0e7e3130b admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:22:41.091352 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86eeb11a-9003-4eb6-94a6-ab001fdafce4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:22:41.091956 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86eeb11a-9003-4eb6-94a6-ab001fdafce4 admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:41.092265 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86eeb11a-9003-4eb6-94a6-ab001fdafce4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:41.092538 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86eeb11a-9003-4eb6-94a6-ab001fdafce4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:41.093224 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86eeb11a-9003-4eb6-94a6-ab001fdafce4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:22:41.094796 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86eeb11a-9003-4eb6-94a6-ab001fdafce4 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:22:41.104870 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1141/2253] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Sep 23 15:22:41 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:22:41.109200 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92976cc9-f98c-4da2-9859-715c64b7640e admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:22:41.120526 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92976cc9-f98c-4da2-9859-715c64b7640e admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:22:41.127007 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8a49a9-9481-4cb6-84c6-bc7e395847f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:22:41.127670 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8a49a9-9481-4cb6-84c6-bc7e395847f2 admin admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:41.129714 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8a49a9-9481-4cb6-84c6-bc7e395847f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:41.129714 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8a49a9-9481-4cb6-84c6-bc7e395847f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:41.129714 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8a49a9-9481-4cb6-84c6-bc7e395847f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:22:41.130911 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8a49a9-9481-4cb6-84c6-bc7e395847f2 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:22:41.143176 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1113/2254] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Sep 23 15:22:41 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:22:41.147698 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86eeb11a-9003-4eb6-94a6-ab001fdafce4 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:22:41.157941 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86eeb11a-9003-4eb6-94a6-ab001fdafce4 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:22:41.162596 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad18e67-8647-4433-af2b-7bea8e865c53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:22:41.163247 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad18e67-8647-4433-af2b-7bea8e865c53 admin admin] REQUEST_METHOD: `PUT` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:41.164290 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad18e67-8647-4433-af2b-7bea8e865c53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:41.164290 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad18e67-8647-4433-af2b-7bea8e865c53 admin admin] PATH_INFO: `/v3/projects/20075491d44d4be7a541cf92ecc722ee/users/3ff015df35f44bf3a46355350499039c/roles/af257be2dee54d6b942f58650196582c` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:41.180042 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad18e67-8647-4433-af2b-7bea8e865c53 admin admin] RBAC: Authorizing `identity:create_grant(project_id=20075491d44d4be7a541cf92ecc722ee, user_id=3ff015df35f44bf3a46355350499039c, role_id=af257be2dee54d6b942f58650196582c)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:22:41.182731 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad18e67-8647-4433-af2b-7bea8e865c53 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:22:41.356184 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1142/2255] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Sep 23 15:22:41 2022] PUT /identity/v3/projects/20075491d44d4be7a541cf92ecc722ee/users/3ff015df35f44bf3a46355350499039c/roles/af257be2dee54d6b942f58650196582c => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:22:41.367064 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3965b30a-f923-401b-b01e-02e2e4e7fdc7 None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:41.367360 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3965b30a-f923-401b-b01e-02e2e4e7fdc7 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:41.367721 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3965b30a-f923-401b-b01e-02e2e4e7fdc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:41.397286 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-3965b30a-f923-401b-b01e-02e2e4e7fdc7 None None] MFA Rules not processed for user `3ff015df35f44bf3a46355350499039c`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:22:41.398607 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-3965b30a-f923-401b-b01e-02e2e4e7fdc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:22:41.413861 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76dc94a-a816-4bcd-a801-03dbc2e0965f None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:41.414146 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76dc94a-a816-4bcd-a801-03dbc2e0965f None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:41.414422 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76dc94a-a816-4bcd-a801-03dbc2e0965f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:41.457386 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-a76dc94a-a816-4bcd-a801-03dbc2e0965f None None] MFA Rules not processed for user `ee65edb3211b4b71a771594a90a342ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:22:41.458992 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-a76dc94a-a816-4bcd-a801-03dbc2e0965f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:22:41.480548 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1114/2256] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:22:41 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:22:41.495113 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3965b30a-f923-401b-b01e-02e2e4e7fdc7 None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:22:41.496791 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-3965b30a-f923-401b-b01e-02e2e4e7fdc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:22:41.548076 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1143/2257] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:22:41 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:22:41.560430 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76dc94a-a816-4bcd-a801-03dbc2e0965f None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:22:41.562876 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-a76dc94a-a816-4bcd-a801-03dbc2e0965f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:22:41.569436 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3965b30a-f923-401b-b01e-02e2e4e7fdc7 None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:22:41.573173 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddd6532-d2ba-4ccd-b67c-04a47c230739 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51c39f1d05634589a88c0b6477bd014e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:22:41.573953 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddd6532-d2ba-4ccd-b67c-04a47c230739 service glance] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:41.574237 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddd6532-d2ba-4ccd-b67c-04a47c230739 service glance] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:41.574511 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddd6532-d2ba-4ccd-b67c-04a47c230739 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:41.590754 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddd6532-d2ba-4ccd-b67c-04a47c230739 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:22:41.594108 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddd6532-d2ba-4ccd-b67c-04a47c230739 service glance] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:22:41.606583 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1115/2258] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:22:41 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 23 15:22:41.636318 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76dc94a-a816-4bcd-a801-03dbc2e0965f None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:22:41.641036 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d951692-621e-4ea1-8626-28c43c2a082c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feca6278be904ef8b0dc580a67e477f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:22:41.641733 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d951692-621e-4ea1-8626-28c43c2a082c service cinder] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:41.642084 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d951692-621e-4ea1-8626-28c43c2a082c service cinder] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:41.642430 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d951692-621e-4ea1-8626-28c43c2a082c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:41.651332 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d951692-621e-4ea1-8626-28c43c2a082c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:22:41.652768 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d951692-621e-4ea1-8626-28c43c2a082c service cinder] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:22:41.661917 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1144/2259] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:22:41 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 23 15:22:41.745194 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:22:44.529715 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8c5179-c122-4c5d-a10b-30adab2c9523 None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:44.538328 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8c5179-c122-4c5d-a10b-30adab2c9523 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:44.538328 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8c5179-c122-4c5d-a10b-30adab2c9523 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:44.579120 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-eb8c5179-c122-4c5d-a10b-30adab2c9523 None None] MFA Rules not processed for user `ee65edb3211b4b71a771594a90a342ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:22:44.580831 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-eb8c5179-c122-4c5d-a10b-30adab2c9523 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:22:44.601179 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1116/2260] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:22:44 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:22:45.315377 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:22:45.561168 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:22:46.731851 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:22:46.921847 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:22:47.332353 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:22:47.813080 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:22:48.240200 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:22:48.401159 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:22:48.624388 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:22:49.063939 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:22:49.293031 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:22:49.404400 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d951692-621e-4ea1-8626-28c43c2a082c service cinder] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:22:49.406220 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-8d951692-621e-4ea1-8626-28c43c2a082c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:22:49.474298 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d951692-621e-4ea1-8626-28c43c2a082c service cinder] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:22:49.478922 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8301cd89-d40d-4cf9-88bb-15c4944a53fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:22:49.479614 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8301cd89-d40d-4cf9-88bb-15c4944a53fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:49.479899 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8301cd89-d40d-4cf9-88bb-15c4944a53fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:49.480210 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8301cd89-d40d-4cf9-88bb-15c4944a53fa admin admin] PATH_INFO: `/v3/users/664fc815c345413a95a9b3f38c976274` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:49.481355 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8301cd89-d40d-4cf9-88bb-15c4944a53fa admin admin] RBAC: Authorizing `identity:delete_user(user_id=664fc815c345413a95a9b3f38c976274)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:22:49.483648 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8301cd89-d40d-4cf9-88bb-15c4944a53fa admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:22:49.771401 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-8301cd89-d40d-4cf9-88bb-15c4944a53fa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '664fc815c345413a95a9b3f38c976274'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:22:49.783527 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-8301cd89-d40d-4cf9-88bb-15c4944a53fa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '664fc815c345413a95a9b3f38c976274'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:22:49.790715 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-8301cd89-d40d-4cf9-88bb-15c4944a53fa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '664fc815c345413a95a9b3f38c976274'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:22:49.921954 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-8301cd89-d40d-4cf9-88bb-15c4944a53fa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '664fc815c345413a95a9b3f38c976274'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:22:49.923229 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-8301cd89-d40d-4cf9-88bb-15c4944a53fa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '664fc815c345413a95a9b3f38c976274'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:22:49.930417 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1145/2261] 10.222.0.55 () {58 vars in 1224 bytes} [Fri Sep 23 15:22:49 2022] DELETE /identity/v3/users/664fc815c345413a95a9b3f38c976274 => generated 0 bytes in 527 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:22:49.935134 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edef305-6830-440d-abd9-af95cf1de6c2 None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:49.935429 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edef305-6830-440d-abd9-af95cf1de6c2 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:49.936549 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edef305-6830-440d-abd9-af95cf1de6c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:49.958452 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-0edef305-6830-440d-abd9-af95cf1de6c2 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:22:49.959795 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-0edef305-6830-440d-abd9-af95cf1de6c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:22:50.030782 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1117/2262] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:22:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:22:50.041300 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8301cd89-d40d-4cf9-88bb-15c4944a53fa admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:22:50.042716 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-8301cd89-d40d-4cf9-88bb-15c4944a53fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:22:50.116660 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8301cd89-d40d-4cf9-88bb-15c4944a53fa admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:22:50.117605 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f94c89-c6ca-4779-a7b0-6916e6fc6f36 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd259178570864ffeb936ee10dfec6b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:22:50.118217 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f94c89-c6ca-4779-a7b0-6916e6fc6f36 service neutron] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:50.118537 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f94c89-c6ca-4779-a7b0-6916e6fc6f36 service neutron] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:50.118841 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f94c89-c6ca-4779-a7b0-6916e6fc6f36 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:50.128561 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f94c89-c6ca-4779-a7b0-6916e6fc6f36 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:22:50.130064 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f94c89-c6ca-4779-a7b0-6916e6fc6f36 service neutron] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:22:50.141311 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1146/2263] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:22:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:22:50.475724 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:22:50.664296 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:22:50.787295 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edef305-6830-440d-abd9-af95cf1de6c2 None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:22:50.788929 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-0edef305-6830-440d-abd9-af95cf1de6c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:22:50.802975 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edef305-6830-440d-abd9-af95cf1de6c2 None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:22:50.818554 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997a34b5-a73a-495f-86c5-8b1ec5ce1a7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:22:50.819412 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997a34b5-a73a-495f-86c5-8b1ec5ce1a7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:50.819789 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997a34b5-a73a-495f-86c5-8b1ec5ce1a7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:50.820203 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997a34b5-a73a-495f-86c5-8b1ec5ce1a7c admin admin] PATH_INFO: `/v3/projects/5ff374b0f9b640009a325a14d202a6a8` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:50.821499 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997a34b5-a73a-495f-86c5-8b1ec5ce1a7c admin admin] RBAC: Authorizing `identity:delete_project(project_id=5ff374b0f9b640009a325a14d202a6a8)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:22:50.824556 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997a34b5-a73a-495f-86c5-8b1ec5ce1a7c admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:22:50.955408 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-997a34b5-a73a-495f-86c5-8b1ec5ce1a7c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5ff374b0f9b640009a325a14d202a6a8', 'request_id': 'req-997a34b5-a73a-495f-86c5-8b1ec5ce1a7c'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:22:50.960900 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-997a34b5-a73a-495f-86c5-8b1ec5ce1a7c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5ff374b0f9b640009a325a14d202a6a8', 'request_id': 'req-997a34b5-a73a-495f-86c5-8b1ec5ce1a7c'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:22:51.114681 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-997a34b5-a73a-495f-86c5-8b1ec5ce1a7c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5ff374b0f9b640009a325a14d202a6a8', 'request_id': 'req-997a34b5-a73a-495f-86c5-8b1ec5ce1a7c'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:22:51.128305 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-997a34b5-a73a-495f-86c5-8b1ec5ce1a7c admin admin] The token cache is being invalidate because project 5ff374b0f9b640009a325a14d202a6a8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81993) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 15:22:51.128608 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-997a34b5-a73a-495f-86c5-8b1ec5ce1a7c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:22:51.129235 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-997a34b5-a73a-495f-86c5-8b1ec5ce1a7c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:22:51.131271 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1118/2264] 10.222.0.55 () {58 vars in 1233 bytes} [Fri Sep 23 15:22:50 2022] DELETE /identity/v3/projects/5ff374b0f9b640009a325a14d202a6a8 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:22:51.136401 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc47657-2a88-4bbf-805b-e3574933d4ea None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:51.136683 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc47657-2a88-4bbf-805b-e3574933d4ea None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:51.136955 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc47657-2a88-4bbf-805b-e3574933d4ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:51.160285 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-abc47657-2a88-4bbf-805b-e3574933d4ea None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:22:51.161741 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-abc47657-2a88-4bbf-805b-e3574933d4ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:22:51.230059 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1147/2265] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:22:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:22:51.240376 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8a1bc4-2353-429a-a1ca-71a8cca628e9 None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:51.241711 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8a1bc4-2353-429a-a1ca-71a8cca628e9 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:51.241711 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8a1bc4-2353-429a-a1ca-71a8cca628e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:51.262646 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-fe8a1bc4-2353-429a-a1ca-71a8cca628e9 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:22:51.264387 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-fe8a1bc4-2353-429a-a1ca-71a8cca628e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:22:51.272660 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1119/2266] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:22:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:22:51.277671 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc47657-2a88-4bbf-805b-e3574933d4ea None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:22:51.288024 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc47657-2a88-4bbf-805b-e3574933d4ea None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:22:51.292722 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae733558-fc6c-4667-8962-a1c1bbc7346b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:22:51.293417 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae733558-fc6c-4667-8962-a1c1bbc7346b admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:51.293603 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae733558-fc6c-4667-8962-a1c1bbc7346b admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:51.293889 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae733558-fc6c-4667-8962-a1c1bbc7346b admin admin] PATH_INFO: `/v3/domains` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:51.294557 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae733558-fc6c-4667-8962-a1c1bbc7346b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:22:51.294863 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae733558-fc6c-4667-8962-a1c1bbc7346b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:22:51.298050 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae733558-fc6c-4667-8962-a1c1bbc7346b admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:22:51.315853 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1148/2267] 10.222.0.55 () {58 vars in 1153 bytes} [Fri Sep 23 15:22:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:22:51.317018 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8a1bc4-2353-429a-a1ca-71a8cca628e9 None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:22:51.327801 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8a1bc4-2353-429a-a1ca-71a8cca628e9 None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:22:51.333075 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2622d4d-d362-4b5f-95d6-08663e7e4331 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:22:51.333728 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2622d4d-d362-4b5f-95d6-08663e7e4331 admin admin] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:51.334009 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2622d4d-d362-4b5f-95d6-08663e7e4331 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:51.334287 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2622d4d-d362-4b5f-95d6-08663e7e4331 admin admin] PATH_INFO: `/v3/projects` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:51.334917 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2622d4d-d362-4b5f-95d6-08663e7e4331 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:22:51.337125 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2622d4d-d362-4b5f-95d6-08663e7e4331 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:22:51.508678 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1120/2268] 10.222.0.55 () {60 vars in 1153 bytes} [Fri Sep 23 15:22:51 2022] POST /identity/v3/projects => generated 377 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:22:51.509913 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae733558-fc6c-4667-8962-a1c1bbc7346b admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:22:51.522577 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae733558-fc6c-4667-8962-a1c1bbc7346b admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:22:51.613607 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a01aa27-520d-4fb2-9ab7-11d0bc491aa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:22:51.614228 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a01aa27-520d-4fb2-9ab7-11d0bc491aa1 admin admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:51.614511 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a01aa27-520d-4fb2-9ab7-11d0bc491aa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:51.614781 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a01aa27-520d-4fb2-9ab7-11d0bc491aa1 admin admin] PATH_INFO: `/v3/users` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:51.615422 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a01aa27-520d-4fb2-9ab7-11d0bc491aa1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:22:51.617415 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a01aa27-520d-4fb2-9ab7-11d0bc491aa1 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:22:51.751471 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:22:51.758647 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1149/2269] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Sep 23 15:22:51 2022] POST /identity/v3/users => generated 289 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:22:51.762394 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2622d4d-d362-4b5f-95d6-08663e7e4331 admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:22:51.774272 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2622d4d-d362-4b5f-95d6-08663e7e4331 admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:22:51.780918 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffe33a0-58f7-4d1d-bf13-bf8d37724dde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:22:51.781395 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffe33a0-58f7-4d1d-bf13-bf8d37724dde admin admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:51.781724 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffe33a0-58f7-4d1d-bf13-bf8d37724dde admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:51.782206 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffe33a0-58f7-4d1d-bf13-bf8d37724dde admin admin] PATH_INFO: `/v3/roles` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:51.782856 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffe33a0-58f7-4d1d-bf13-bf8d37724dde admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:22:51.784791 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffe33a0-58f7-4d1d-bf13-bf8d37724dde admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:22:51.799678 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1121/2270] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Sep 23 15:22:51 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:22:51.803434 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a01aa27-520d-4fb2-9ab7-11d0bc491aa1 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:22:51.821926 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a01aa27-520d-4fb2-9ab7-11d0bc491aa1 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:22:51.828434 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181f33dd-1e82-4e33-af19-9e27b9e8357b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:22:51.829402 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181f33dd-1e82-4e33-af19-9e27b9e8357b admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:51.829620 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181f33dd-1e82-4e33-af19-9e27b9e8357b admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:51.830065 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181f33dd-1e82-4e33-af19-9e27b9e8357b admin admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:51.830786 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181f33dd-1e82-4e33-af19-9e27b9e8357b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:22:51.832655 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181f33dd-1e82-4e33-af19-9e27b9e8357b admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:22:51.851439 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1150/2271] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Sep 23 15:22:51 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:22:51.852509 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffe33a0-58f7-4d1d-bf13-bf8d37724dde admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:22:51.867310 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffe33a0-58f7-4d1d-bf13-bf8d37724dde admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:22:51.873822 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b736ad9c-ab22-4093-aea2-a92e1a3d4e1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:22:51.874401 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b736ad9c-ab22-4093-aea2-a92e1a3d4e1c admin admin] REQUEST_METHOD: `PUT` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:51.874685 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b736ad9c-ab22-4093-aea2-a92e1a3d4e1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:51.874960 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b736ad9c-ab22-4093-aea2-a92e1a3d4e1c admin admin] PATH_INFO: `/v3/projects/da5d992d91c34ac99c928ec3fedaf320/users/866977f6bf764a3d8805174add5703fb/roles/af257be2dee54d6b942f58650196582c` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:51.911488 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b736ad9c-ab22-4093-aea2-a92e1a3d4e1c admin admin] RBAC: Authorizing `identity:create_grant(project_id=da5d992d91c34ac99c928ec3fedaf320, user_id=866977f6bf764a3d8805174add5703fb, role_id=af257be2dee54d6b942f58650196582c)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:22:51.914366 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b736ad9c-ab22-4093-aea2-a92e1a3d4e1c admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:22:52.031192 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1122/2272] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Sep 23 15:22:51 2022] PUT /identity/v3/projects/da5d992d91c34ac99c928ec3fedaf320/users/866977f6bf764a3d8805174add5703fb/roles/af257be2dee54d6b942f58650196582c => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:22:52.043082 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2856191e-172a-4461-a0c0-47b51384dd54 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:52.043082 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2856191e-172a-4461-a0c0-47b51384dd54 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:52.043082 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2856191e-172a-4461-a0c0-47b51384dd54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:52.080714 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-2856191e-172a-4461-a0c0-47b51384dd54 None None] MFA Rules not processed for user `866977f6bf764a3d8805174add5703fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:22:52.082054 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-2856191e-172a-4461-a0c0-47b51384dd54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:22:52.109615 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:22:52.162535 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1151/2273] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:22:52 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:22:52.167010 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b736ad9c-ab22-4093-aea2-a92e1a3d4e1c admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:22:52.237174 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b736ad9c-ab22-4093-aea2-a92e1a3d4e1c admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:22:52.242024 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f5d08e-5886-41d6-b5d5-219b0d14f2d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:22:52.242693 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f5d08e-5886-41d6-b5d5-219b0d14f2d2 admin admin] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:52.242978 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f5d08e-5886-41d6-b5d5-219b0d14f2d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:52.243256 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f5d08e-5886-41d6-b5d5-219b0d14f2d2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:52.243898 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f5d08e-5886-41d6-b5d5-219b0d14f2d2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:22:52.245844 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f5d08e-5886-41d6-b5d5-219b0d14f2d2 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:22:52.351808 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1123/2274] 10.222.0.55 () {60 vars in 1153 bytes} [Fri Sep 23 15:22:52 2022] POST /identity/v3/projects => generated 375 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:22:52.363189 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2856191e-172a-4461-a0c0-47b51384dd54 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:22:52.377784 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2856191e-172a-4461-a0c0-47b51384dd54 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:22:52.443508 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47b7251-e391-4772-92a5-ae111599aa5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:22:52.444575 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47b7251-e391-4772-92a5-ae111599aa5d admin admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:52.445030 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47b7251-e391-4772-92a5-ae111599aa5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:52.445313 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47b7251-e391-4772-92a5-ae111599aa5d admin admin] PATH_INFO: `/v3/users` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:52.446077 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47b7251-e391-4772-92a5-ae111599aa5d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:22:52.448157 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47b7251-e391-4772-92a5-ae111599aa5d admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:22:52.617157 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1152/2275] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Sep 23 15:22:52 2022] POST /identity/v3/users => generated 288 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 23 15:22:52.617390 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f5d08e-5886-41d6-b5d5-219b0d14f2d2 admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:22:52.641584 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f5d08e-5886-41d6-b5d5-219b0d14f2d2 admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:22:52.647665 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1ef451-01fd-412e-8029-d31c3533b242 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:22:52.651549 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1ef451-01fd-412e-8029-d31c3533b242 admin admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:52.651549 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1ef451-01fd-412e-8029-d31c3533b242 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:52.651549 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1ef451-01fd-412e-8029-d31c3533b242 admin admin] PATH_INFO: `/v3/roles` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:52.651549 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1ef451-01fd-412e-8029-d31c3533b242 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:22:52.652183 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1ef451-01fd-412e-8029-d31c3533b242 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:22:52.666197 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1124/2276] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Sep 23 15:22:52 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:22:52.670604 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47b7251-e391-4772-92a5-ae111599aa5d admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:22:52.682053 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47b7251-e391-4772-92a5-ae111599aa5d admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:22:52.687776 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c0b425-7103-4433-bd58-2a88338e16dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:22:52.688475 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c0b425-7103-4433-bd58-2a88338e16dc admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:52.688763 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c0b425-7103-4433-bd58-2a88338e16dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:52.689040 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c0b425-7103-4433-bd58-2a88338e16dc admin admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:52.689721 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c0b425-7103-4433-bd58-2a88338e16dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:22:52.691496 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c0b425-7103-4433-bd58-2a88338e16dc admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:22:52.706002 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1153/2277] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Sep 23 15:22:52 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:22:52.707224 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1ef451-01fd-412e-8029-d31c3533b242 admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:22:52.710803 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:22:52.720068 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1ef451-01fd-412e-8029-d31c3533b242 admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:22:52.725413 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a8fc18-874d-4f6c-8ecd-49ca547dd7ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:22:52.726089 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a8fc18-874d-4f6c-8ecd-49ca547dd7ca admin admin] REQUEST_METHOD: `PUT` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:52.726390 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a8fc18-874d-4f6c-8ecd-49ca547dd7ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:52.726668 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a8fc18-874d-4f6c-8ecd-49ca547dd7ca admin admin] PATH_INFO: `/v3/projects/435f82ccb24c40e6b7dedc2e5cb15174/users/d9538a9e1d8741d3a1bb1dfd3383b47d/roles/af257be2dee54d6b942f58650196582c` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:52.754731 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a8fc18-874d-4f6c-8ecd-49ca547dd7ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=435f82ccb24c40e6b7dedc2e5cb15174, user_id=d9538a9e1d8741d3a1bb1dfd3383b47d, role_id=af257be2dee54d6b942f58650196582c)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:22:52.755728 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a8fc18-874d-4f6c-8ecd-49ca547dd7ca admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:22:52.870029 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1125/2278] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Sep 23 15:22:52 2022] PUT /identity/v3/projects/435f82ccb24c40e6b7dedc2e5cb15174/users/d9538a9e1d8741d3a1bb1dfd3383b47d/roles/af257be2dee54d6b942f58650196582c => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:22:52.882713 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bf8598-2a92-411b-8586-cc457dd92a1e None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:52.883118 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bf8598-2a92-411b-8586-cc457dd92a1e None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:52.883326 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bf8598-2a92-411b-8586-cc457dd92a1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:52.906372 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9748dd8e-fc3a-4c1d-bf8a-e60a178bbe5c None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:52.906733 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9748dd8e-fc3a-4c1d-bf8a-e60a178bbe5c None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:52.909263 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9748dd8e-fc3a-4c1d-bf8a-e60a178bbe5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:52.932208 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-9748dd8e-fc3a-4c1d-bf8a-e60a178bbe5c None None] MFA Rules not processed for user `68206a6b4f244d45bf7bf1504396e430`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:22:52.938134 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-99bf8598-2a92-411b-8586-cc457dd92a1e None None] MFA Rules not processed for user `d9538a9e1d8741d3a1bb1dfd3383b47d`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:22:52.938476 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-9748dd8e-fc3a-4c1d-bf8a-e60a178bbe5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:22:52.938587 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-99bf8598-2a92-411b-8586-cc457dd92a1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:22:53.003045 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1126/2279] 10.222.0.55 () {58 vars in 1009 bytes} [Fri Sep 23 15:22:52 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:22:53.048064 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1154/2280] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:22:52 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:22:53.051048 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9748dd8e-fc3a-4c1d-bf8a-e60a178bbe5c None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:22:53.060461 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bf8598-2a92-411b-8586-cc457dd92a1e None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:22:53.068596 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-99bf8598-2a92-411b-8586-cc457dd92a1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:22:53.140970 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9748dd8e-fc3a-4c1d-bf8a-e60a178bbe5c None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:22:53.146059 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0ee31a-249b-44e4-8d15-0ef6271cc264 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:22:53.146724 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0ee31a-249b-44e4-8d15-0ef6271cc264 admin admin] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:53.147193 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0ee31a-249b-44e4-8d15-0ef6271cc264 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:53.147318 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0ee31a-249b-44e4-8d15-0ef6271cc264 admin admin] PATH_INFO: `/v3/projects` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:53.147901 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0ee31a-249b-44e4-8d15-0ef6271cc264 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:22:53.150572 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0ee31a-249b-44e4-8d15-0ef6271cc264 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:22:53.167541 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bf8598-2a92-411b-8586-cc457dd92a1e None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:22:53.173359 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8983f31e-2165-46cc-b824-968be4691262 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2228c0b49a0249bbaf524a8c4e9e108b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:22:53.173977 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8983f31e-2165-46cc-b824-968be4691262 service swift] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:53.174261 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8983f31e-2165-46cc-b824-968be4691262 service swift] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:53.174537 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8983f31e-2165-46cc-b824-968be4691262 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:53.183136 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8983f31e-2165-46cc-b824-968be4691262 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:22:53.184776 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8983f31e-2165-46cc-b824-968be4691262 service swift] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:22:53.194012 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1155/2281] 10.222.0.55 () {60 vars in 1384 bytes} [Fri Sep 23 15:22:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 23 15:22:53.264206 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:22:53.319084 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1127/2282] 10.222.0.55 () {60 vars in 1153 bytes} [Fri Sep 23 15:22:53 2022] POST /identity/v3/projects => generated 375 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:22:53.322637 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8983f31e-2165-46cc-b824-968be4691262 service swift] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:22:53.337248 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8983f31e-2165-46cc-b824-968be4691262 service swift] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:22:53.430737 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f88a20-400f-477a-8130-5d53bc0f4585 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:22:53.431755 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f88a20-400f-477a-8130-5d53bc0f4585 admin admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:53.432306 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f88a20-400f-477a-8130-5d53bc0f4585 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:53.433169 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f88a20-400f-477a-8130-5d53bc0f4585 admin admin] PATH_INFO: `/v3/users` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:53.434309 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f88a20-400f-477a-8130-5d53bc0f4585 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:22:53.437860 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f88a20-400f-477a-8130-5d53bc0f4585 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:22:53.610057 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:22:53.736628 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1156/2283] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Sep 23 15:22:53 2022] POST /identity/v3/users => generated 294 bytes in 414 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:22:53.740625 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0ee31a-249b-44e4-8d15-0ef6271cc264 admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:22:53.758153 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0ee31a-249b-44e4-8d15-0ef6271cc264 admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:22:53.760510 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49a7102-d89f-4688-8050-beea7e743a02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:22:53.761180 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49a7102-d89f-4688-8050-beea7e743a02 admin admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:53.761724 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49a7102-d89f-4688-8050-beea7e743a02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:53.761724 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49a7102-d89f-4688-8050-beea7e743a02 admin admin] PATH_INFO: `/v3/roles` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:53.762428 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49a7102-d89f-4688-8050-beea7e743a02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:22:53.764164 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49a7102-d89f-4688-8050-beea7e743a02 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:22:53.778601 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1128/2284] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Sep 23 15:22:53 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:22:53.784034 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f88a20-400f-477a-8130-5d53bc0f4585 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:22:53.793856 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f88a20-400f-477a-8130-5d53bc0f4585 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:22:53.800262 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4884db22-7e21-4d7d-978f-fa0d2f4678c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:22:53.801951 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4884db22-7e21-4d7d-978f-fa0d2f4678c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:53.804599 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4884db22-7e21-4d7d-978f-fa0d2f4678c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:53.804948 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4884db22-7e21-4d7d-978f-fa0d2f4678c6 admin admin] PATH_INFO: `/v3/projects/6dc7f4d2c9cc45d780ef4d56d1c1964e/users/b645586fc8d140cc969cff59b63c44b6/roles/8e2919e79df040e8926f21607818b58c` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:53.826586 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4884db22-7e21-4d7d-978f-fa0d2f4678c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6dc7f4d2c9cc45d780ef4d56d1c1964e, user_id=b645586fc8d140cc969cff59b63c44b6, role_id=8e2919e79df040e8926f21607818b58c)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:22:53.829983 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4884db22-7e21-4d7d-978f-fa0d2f4678c6 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:22:54.002133 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:22:54.012678 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1157/2285] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Sep 23 15:22:53 2022] PUT /identity/v3/projects/6dc7f4d2c9cc45d780ef4d56d1c1964e/users/b645586fc8d140cc969cff59b63c44b6/roles/8e2919e79df040e8926f21607818b58c => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:22:54.019309 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49a7102-d89f-4688-8050-beea7e743a02 admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:22:54.040014 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49a7102-d89f-4688-8050-beea7e743a02 admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:22:54.132239 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdef1a42-65e5-4a0d-9549-440dc4534d9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:22:54.132855 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdef1a42-65e5-4a0d-9549-440dc4534d9c admin admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:54.133135 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdef1a42-65e5-4a0d-9549-440dc4534d9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:54.133447 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdef1a42-65e5-4a0d-9549-440dc4534d9c admin admin] PATH_INFO: `/v3/roles` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:54.134095 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdef1a42-65e5-4a0d-9549-440dc4534d9c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:22:54.135714 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdef1a42-65e5-4a0d-9549-440dc4534d9c admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:22:54.148029 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1129/2286] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Sep 23 15:22:54 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:22:54.153170 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4884db22-7e21-4d7d-978f-fa0d2f4678c6 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:22:54.175374 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4884db22-7e21-4d7d-978f-fa0d2f4678c6 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:22:54.179512 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053b6b08-9015-4fcb-82a9-052fd75a59cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:22:54.180341 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053b6b08-9015-4fcb-82a9-052fd75a59cd admin admin] REQUEST_METHOD: `PUT` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:54.180633 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053b6b08-9015-4fcb-82a9-052fd75a59cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:54.180903 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053b6b08-9015-4fcb-82a9-052fd75a59cd admin admin] PATH_INFO: `/v3/projects/6dc7f4d2c9cc45d780ef4d56d1c1964e/users/b645586fc8d140cc969cff59b63c44b6/roles/8e2919e79df040e8926f21607818b58c` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:54.183173 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053b6b08-9015-4fcb-82a9-052fd75a59cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=6dc7f4d2c9cc45d780ef4d56d1c1964e, user_id=b645586fc8d140cc969cff59b63c44b6, role_id=8e2919e79df040e8926f21607818b58c)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:22:54.187597 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053b6b08-9015-4fcb-82a9-052fd75a59cd admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:22:54.199465 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1158/2287] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Sep 23 15:22:54 2022] PUT /identity/v3/projects/6dc7f4d2c9cc45d780ef4d56d1c1964e/users/b645586fc8d140cc969cff59b63c44b6/roles/8e2919e79df040e8926f21607818b58c => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:22:54.211465 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02b5ff1-8a76-45c4-a37b-02a62bd794c6 None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:54.211753 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02b5ff1-8a76-45c4-a37b-02a62bd794c6 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:54.212083 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02b5ff1-8a76-45c4-a37b-02a62bd794c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:54.259275 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-f02b5ff1-8a76-45c4-a37b-02a62bd794c6 None None] MFA Rules not processed for user `b645586fc8d140cc969cff59b63c44b6`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:22:54.260653 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-f02b5ff1-8a76-45c4-a37b-02a62bd794c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:22:54.366004 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1130/2288] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:22:54 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:22:54.378715 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053b6b08-9015-4fcb-82a9-052fd75a59cd admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:22:54.381661 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-053b6b08-9015-4fcb-82a9-052fd75a59cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:22:54.449510 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053b6b08-9015-4fcb-82a9-052fd75a59cd admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:22:54.453036 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:22:54.454067 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678b9302-e648-4412-ab73-3dcad25d5e6a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feca6278be904ef8b0dc580a67e477f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:22:54.454680 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678b9302-e648-4412-ab73-3dcad25d5e6a service cinder] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:54.454955 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678b9302-e648-4412-ab73-3dcad25d5e6a service cinder] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:54.455251 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678b9302-e648-4412-ab73-3dcad25d5e6a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:54.515833 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:22:54.527744 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678b9302-e648-4412-ab73-3dcad25d5e6a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:22:54.529235 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678b9302-e648-4412-ab73-3dcad25d5e6a service cinder] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:22:54.540909 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1159/2289] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:22:54 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:22:55.264618 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:22:55.337739 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:22:55.726394 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:22:56.057973 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:22:56.286597 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:22:59.393631 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02b5ff1-8a76-45c4-a37b-02a62bd794c6 None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:22:59.398479 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-f02b5ff1-8a76-45c4-a37b-02a62bd794c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:22:59.432813 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02b5ff1-8a76-45c4-a37b-02a62bd794c6 None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:22:59.440176 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561bd2fb-3f9e-485a-8fa0-b24aee6c3c00 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feca6278be904ef8b0dc580a67e477f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:22:59.442189 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561bd2fb-3f9e-485a-8fa0-b24aee6c3c00 service cinder] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:59.442697 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561bd2fb-3f9e-485a-8fa0-b24aee6c3c00 service cinder] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:59.443176 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561bd2fb-3f9e-485a-8fa0-b24aee6c3c00 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:59.489466 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:22:59.581521 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561bd2fb-3f9e-485a-8fa0-b24aee6c3c00 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:22:59.587008 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561bd2fb-3f9e-485a-8fa0-b24aee6c3c00 service cinder] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:22:59.607682 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1131/2290] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:22:59 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:23:02.823571 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678b9302-e648-4412-ab73-3dcad25d5e6a service cinder] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:23:02.835519 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678b9302-e648-4412-ab73-3dcad25d5e6a service cinder] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:23:02.839850 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2973d3-0b27-492a-ad50-80af82fd9309 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feca6278be904ef8b0dc580a67e477f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:23:02.840903 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2973d3-0b27-492a-ad50-80af82fd9309 service cinder] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:23:02.841546 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2973d3-0b27-492a-ad50-80af82fd9309 service cinder] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:23:02.842109 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2973d3-0b27-492a-ad50-80af82fd9309 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:23:02.853848 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2973d3-0b27-492a-ad50-80af82fd9309 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:23:02.855561 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2973d3-0b27-492a-ad50-80af82fd9309 service cinder] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:23:02.892867 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1160/2291] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:23:02 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:23:07.125050 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9d1cdc-cebd-4b95-817e-b2fe37031bf1 None None] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:23:07.126202 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9d1cdc-cebd-4b95-817e-b2fe37031bf1 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:23:07.126547 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9d1cdc-cebd-4b95-817e-b2fe37031bf1 None None] PATH_INFO: `/` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:23:07.128118 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1132/2292] 10.222.0.55 () {54 vars in 961 bytes} [Fri Sep 23 15:23:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:23:07.136308 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e3c081-6a48-464e-b8e1-51579cf636d6 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:23:07.139097 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e3c081-6a48-464e-b8e1-51579cf636d6 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:23:07.139097 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e3c081-6a48-464e-b8e1-51579cf636d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:23:07.164852 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-91e3c081-6a48-464e-b8e1-51579cf636d6 None None] MFA Rules not processed for user `09f1a55d180642c781a6e0b017d9f0aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:23:07.164852 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-91e3c081-6a48-464e-b8e1-51579cf636d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:23:07.277254 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1161/2293] 10.222.0.55 () {58 vars in 1058 bytes} [Fri Sep 23 15:23:07 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:23:07.288542 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9d1cdc-cebd-4b95-817e-b2fe37031bf1 None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:23:07.318400 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9d1cdc-cebd-4b95-817e-b2fe37031bf1 None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:23:07.324423 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657e40cc-78f1-4bb0-a413-3795516690a0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09f1a55d180642c781a6e0b017d9f0aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:23:07.325268 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657e40cc-78f1-4bb0-a413-3795516690a0 service placement] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:23:07.325777 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657e40cc-78f1-4bb0-a413-3795516690a0 service placement] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:23:07.326221 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657e40cc-78f1-4bb0-a413-3795516690a0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:23:07.328217 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-657e40cc-78f1-4bb0-a413-3795516690a0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:23:07.348462 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657e40cc-78f1-4bb0-a413-3795516690a0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:23:07.350247 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657e40cc-78f1-4bb0-a413-3795516690a0 service placement] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:23:07.368054 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1133/2294] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:23:07 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 23 15:23:07.788250 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e3c081-6a48-464e-b8e1-51579cf636d6 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:23:07.790581 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-91e3c081-6a48-464e-b8e1-51579cf636d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:23:07.870816 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e3c081-6a48-464e-b8e1-51579cf636d6 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:23:07.876512 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fb12aa-5c4a-4318-b92e-56cdcab7524b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:23:07.877152 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fb12aa-5c4a-4318-b92e-56cdcab7524b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:23:07.877438 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fb12aa-5c4a-4318-b92e-56cdcab7524b admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:23:07.877714 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fb12aa-5c4a-4318-b92e-56cdcab7524b admin admin] PATH_INFO: `/v3/users/ffd24319a3884c0c965fc611db7d40ae` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:23:07.878862 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fb12aa-5c4a-4318-b92e-56cdcab7524b admin admin] RBAC: Authorizing `identity:delete_user(user_id=ffd24319a3884c0c965fc611db7d40ae)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:23:07.880825 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fb12aa-5c4a-4318-b92e-56cdcab7524b admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:23:07.885046 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:23:08.265054 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-97fb12aa-5c4a-4318-b92e-56cdcab7524b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ffd24319a3884c0c965fc611db7d40ae'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:23:08.285701 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-97fb12aa-5c4a-4318-b92e-56cdcab7524b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ffd24319a3884c0c965fc611db7d40ae'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:23:08.294948 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-97fb12aa-5c4a-4318-b92e-56cdcab7524b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ffd24319a3884c0c965fc611db7d40ae'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:23:08.435107 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-97fb12aa-5c4a-4318-b92e-56cdcab7524b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ffd24319a3884c0c965fc611db7d40ae'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:23:08.436188 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-97fb12aa-5c4a-4318-b92e-56cdcab7524b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ffd24319a3884c0c965fc611db7d40ae'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:23:08.443402 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1162/2295] 10.222.0.55 () {58 vars in 1224 bytes} [Fri Sep 23 15:23:07 2022] DELETE /identity/v3/users/ffd24319a3884c0c965fc611db7d40ae => generated 0 bytes in 656 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:23:08.449827 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eca6d2d-56c1-4904-8d71-3726c16b6ccf None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:23:08.450918 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eca6d2d-56c1-4904-8d71-3726c16b6ccf None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:23:08.451376 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eca6d2d-56c1-4904-8d71-3726c16b6ccf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:23:08.478085 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-0eca6d2d-56c1-4904-8d71-3726c16b6ccf None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:23:08.480142 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-0eca6d2d-56c1-4904-8d71-3726c16b6ccf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:23:08.551318 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1134/2296] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:23:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:23:08.561009 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fb12aa-5c4a-4318-b92e-56cdcab7524b admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:23:08.562588 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-97fb12aa-5c4a-4318-b92e-56cdcab7524b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:23:08.625992 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eca6d2d-56c1-4904-8d71-3726c16b6ccf None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:23:08.627626 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-0eca6d2d-56c1-4904-8d71-3726c16b6ccf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:23:08.641625 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eca6d2d-56c1-4904-8d71-3726c16b6ccf None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:23:08.646496 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ff421c-f5ef-45cd-8418-52a5fe1c51fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:23:08.646719 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fb12aa-5c4a-4318-b92e-56cdcab7524b admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:23:08.647615 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ff421c-f5ef-45cd-8418-52a5fe1c51fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:23:08.648136 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ff421c-f5ef-45cd-8418-52a5fe1c51fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:23:08.648564 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ff421c-f5ef-45cd-8418-52a5fe1c51fa admin admin] PATH_INFO: `/v3/users/3ff015df35f44bf3a46355350499039c` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:23:08.649926 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ff421c-f5ef-45cd-8418-52a5fe1c51fa admin admin] RBAC: Authorizing `identity:delete_user(user_id=3ff015df35f44bf3a46355350499039c)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:23:08.652060 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ff421c-f5ef-45cd-8418-52a5fe1c51fa admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:23:08.653717 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba22787-b224-425a-b447-efa85f51f2d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd259178570864ffeb936ee10dfec6b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:23:08.654922 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba22787-b224-425a-b447-efa85f51f2d1 service neutron] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:23:08.655238 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba22787-b224-425a-b447-efa85f51f2d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:23:08.656013 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba22787-b224-425a-b447-efa85f51f2d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:23:08.684220 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba22787-b224-425a-b447-efa85f51f2d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:23:08.695893 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba22787-b224-425a-b447-efa85f51f2d1 service neutron] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:23:08.718485 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1163/2297] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:23:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:23:09.051490 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-53ff421c-f5ef-45cd-8418-52a5fe1c51fa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3ff015df35f44bf3a46355350499039c'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:23:09.064447 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-53ff421c-f5ef-45cd-8418-52a5fe1c51fa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3ff015df35f44bf3a46355350499039c'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:23:09.073610 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-53ff421c-f5ef-45cd-8418-52a5fe1c51fa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3ff015df35f44bf3a46355350499039c'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:23:09.194119 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-53ff421c-f5ef-45cd-8418-52a5fe1c51fa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3ff015df35f44bf3a46355350499039c'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:23:09.194857 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-53ff421c-f5ef-45cd-8418-52a5fe1c51fa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3ff015df35f44bf3a46355350499039c'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:23:09.202298 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1135/2298] 10.222.0.55 () {58 vars in 1224 bytes} [Fri Sep 23 15:23:08 2022] DELETE /identity/v3/users/3ff015df35f44bf3a46355350499039c => generated 0 bytes in 577 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:23:09.206841 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d56d18-34a2-4db4-b6e8-bd04b6a34af5 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:23:09.207141 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d56d18-34a2-4db4-b6e8-bd04b6a34af5 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:23:09.207414 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d56d18-34a2-4db4-b6e8-bd04b6a34af5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:23:09.230514 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-49d56d18-34a2-4db4-b6e8-bd04b6a34af5 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:23:09.231769 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-49d56d18-34a2-4db4-b6e8-bd04b6a34af5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:23:09.309196 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1164/2299] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:23:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:23:09.323833 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ff421c-f5ef-45cd-8418-52a5fe1c51fa admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:23:09.326067 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-53ff421c-f5ef-45cd-8418-52a5fe1c51fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:23:09.358331 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:23:09.408916 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ff421c-f5ef-45cd-8418-52a5fe1c51fa admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:23:09.414333 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8582b118-b905-424f-8ac8-7fc2268c6d57 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd259178570864ffeb936ee10dfec6b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:23:09.415396 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8582b118-b905-424f-8ac8-7fc2268c6d57 service neutron] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:23:09.416019 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8582b118-b905-424f-8ac8-7fc2268c6d57 service neutron] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:23:09.416693 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8582b118-b905-424f-8ac8-7fc2268c6d57 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:23:09.429322 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8582b118-b905-424f-8ac8-7fc2268c6d57 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:23:09.431251 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8582b118-b905-424f-8ac8-7fc2268c6d57 service neutron] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:23:09.448825 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1136/2300] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:23:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:23:09.766827 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d56d18-34a2-4db4-b6e8-bd04b6a34af5 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:23:09.768336 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-49d56d18-34a2-4db4-b6e8-bd04b6a34af5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:23:09.780324 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d56d18-34a2-4db4-b6e8-bd04b6a34af5 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:23:09.784896 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ff952d-07da-4e47-9759-954f9808b3b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:23:09.785527 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ff952d-07da-4e47-9759-954f9808b3b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:23:09.785812 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ff952d-07da-4e47-9759-954f9808b3b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:23:09.786088 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ff952d-07da-4e47-9759-954f9808b3b1 admin admin] PATH_INFO: `/v3/users/866977f6bf764a3d8805174add5703fb` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:23:09.787275 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ff952d-07da-4e47-9759-954f9808b3b1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=866977f6bf764a3d8805174add5703fb)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:23:09.789246 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ff952d-07da-4e47-9759-954f9808b3b1 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:23:09.815040 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:23:09.964119 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8582b118-b905-424f-8ac8-7fc2268c6d57 service neutron] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:23:09.964119 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-8582b118-b905-424f-8ac8-7fc2268c6d57 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:23:09.979890 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8582b118-b905-424f-8ac8-7fc2268c6d57 service neutron] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:23:09.986965 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39fe5c2-7a5b-4953-b749-b8be7a52d6c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:23:09.987845 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39fe5c2-7a5b-4953-b749-b8be7a52d6c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:23:09.988365 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39fe5c2-7a5b-4953-b749-b8be7a52d6c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:23:09.989007 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39fe5c2-7a5b-4953-b749-b8be7a52d6c9 admin admin] PATH_INFO: `/v3/projects/5303d4f32d4c458aba3340b06312cc46` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:23:09.990262 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39fe5c2-7a5b-4953-b749-b8be7a52d6c9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5303d4f32d4c458aba3340b06312cc46)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:23:09.992607 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39fe5c2-7a5b-4953-b749-b8be7a52d6c9 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:23:10.447938 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-36ff952d-07da-4e47-9759-954f9808b3b1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '866977f6bf764a3d8805174add5703fb'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:23:10.454934 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-b39fe5c2-7a5b-4953-b749-b8be7a52d6c9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5303d4f32d4c458aba3340b06312cc46', 'request_id': 'req-b39fe5c2-7a5b-4953-b749-b8be7a52d6c9'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:23:10.462501 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-b39fe5c2-7a5b-4953-b749-b8be7a52d6c9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5303d4f32d4c458aba3340b06312cc46', 'request_id': 'req-b39fe5c2-7a5b-4953-b749-b8be7a52d6c9'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:23:10.465226 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-36ff952d-07da-4e47-9759-954f9808b3b1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '866977f6bf764a3d8805174add5703fb'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:23:10.476804 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-36ff952d-07da-4e47-9759-954f9808b3b1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '866977f6bf764a3d8805174add5703fb'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:23:10.609548 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-b39fe5c2-7a5b-4953-b749-b8be7a52d6c9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5303d4f32d4c458aba3340b06312cc46', 'request_id': 'req-b39fe5c2-7a5b-4953-b749-b8be7a52d6c9'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:23:10.623016 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-b39fe5c2-7a5b-4953-b749-b8be7a52d6c9 admin admin] The token cache is being invalidate because project 5303d4f32d4c458aba3340b06312cc46 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81993) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 15:23:10.623399 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-b39fe5c2-7a5b-4953-b749-b8be7a52d6c9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:23:10.624236 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-b39fe5c2-7a5b-4953-b749-b8be7a52d6c9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:23:10.625854 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1137/2301] 10.222.0.55 () {58 vars in 1233 bytes} [Fri Sep 23 15:23:09 2022] DELETE /identity/v3/projects/5303d4f32d4c458aba3340b06312cc46 => generated 0 bytes in 666 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 15:23:10.631293 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75735d16-90ae-4993-904c-fc33f50fc02c None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:23:10.631700 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75735d16-90ae-4993-904c-fc33f50fc02c None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:23:10.632100 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75735d16-90ae-4993-904c-fc33f50fc02c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:23:10.653423 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-75735d16-90ae-4993-904c-fc33f50fc02c None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:23:10.654784 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-75735d16-90ae-4993-904c-fc33f50fc02c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:23:10.738231 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1138/2302] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:23:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 15:23:10.740236 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d49feff-b5f2-43a0-bc4b-e7851b0e91a8 None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:23:10.740532 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d49feff-b5f2-43a0-bc4b-e7851b0e91a8 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:23:10.740873 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d49feff-b5f2-43a0-bc4b-e7851b0e91a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:23:10.748393 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-36ff952d-07da-4e47-9759-954f9808b3b1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '866977f6bf764a3d8805174add5703fb'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:23:10.749112 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-36ff952d-07da-4e47-9759-954f9808b3b1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '866977f6bf764a3d8805174add5703fb'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:23:10.762525 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1165/2303] 10.222.0.55 () {58 vars in 1224 bytes} [Fri Sep 23 15:23:09 2022] DELETE /identity/v3/users/866977f6bf764a3d8805174add5703fb => generated 0 bytes in 992 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:23:10.765644 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ff952d-07da-4e47-9759-954f9808b3b1 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:23:10.768622 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-36ff952d-07da-4e47-9759-954f9808b3b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:23:10.773826 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-7d49feff-b5f2-43a0-bc4b-e7851b0e91a8 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:23:10.775281 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-7d49feff-b5f2-43a0-bc4b-e7851b0e91a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:23:10.863207 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1139/2304] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:23:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:23:10.864596 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fb1612-ed86-4df4-bb87-a62859ec2d8a None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:23:10.864887 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fb1612-ed86-4df4-bb87-a62859ec2d8a None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:23:10.865163 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fb1612-ed86-4df4-bb87-a62859ec2d8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:23:10.875861 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ff952d-07da-4e47-9759-954f9808b3b1 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:23:10.882648 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0ea919-f087-41b1-9cb1-9e8b48047d87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:23:10.883302 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0ea919-f087-41b1-9cb1-9e8b48047d87 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:23:10.883582 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0ea919-f087-41b1-9cb1-9e8b48047d87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:23:10.884319 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0ea919-f087-41b1-9cb1-9e8b48047d87 admin admin] PATH_INFO: `/v3/projects/20075491d44d4be7a541cf92ecc722ee` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:23:10.885608 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0ea919-f087-41b1-9cb1-9e8b48047d87 admin admin] RBAC: Authorizing `identity:delete_project(project_id=20075491d44d4be7a541cf92ecc722ee)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:23:10.887617 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0ea919-f087-41b1-9cb1-9e8b48047d87 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:23:10.893861 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-57fb1612-ed86-4df4-bb87-a62859ec2d8a None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:23:10.895051 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-57fb1612-ed86-4df4-bb87-a62859ec2d8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:23:10.912295 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1140/2305] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:23:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:23:10.913245 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fb1612-ed86-4df4-bb87-a62859ec2d8a None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:23:10.924131 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fb1612-ed86-4df4-bb87-a62859ec2d8a None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:23:10.929184 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22bff01-b02f-49e4-9995-6ece5e5881e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:23:10.929802 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22bff01-b02f-49e4-9995-6ece5e5881e0 admin admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:23:10.930084 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22bff01-b02f-49e4-9995-6ece5e5881e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:23:10.930356 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22bff01-b02f-49e4-9995-6ece5e5881e0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:23:10.931008 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22bff01-b02f-49e4-9995-6ece5e5881e0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:23:10.931311 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22bff01-b02f-49e4-9995-6ece5e5881e0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:23:10.932948 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22bff01-b02f-49e4-9995-6ece5e5881e0 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:23:10.945510 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1141/2306] 10.222.0.55 () {58 vars in 1153 bytes} [Fri Sep 23 15:23:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:23:10.946423 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22bff01-b02f-49e4-9995-6ece5e5881e0 admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:23:10.948123 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-b22bff01-b02f-49e4-9995-6ece5e5881e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:23:11.007031 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22bff01-b02f-49e4-9995-6ece5e5881e0 admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:23:11.010973 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e81c8a-362f-4d78-9379-a4e50a82a025 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd259178570864ffeb936ee10dfec6b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:23:11.011581 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e81c8a-362f-4d78-9379-a4e50a82a025 service neutron] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:23:11.011862 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e81c8a-362f-4d78-9379-a4e50a82a025 service neutron] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:23:11.012162 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e81c8a-362f-4d78-9379-a4e50a82a025 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:23:11.022088 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e81c8a-362f-4d78-9379-a4e50a82a025 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:23:11.023517 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e81c8a-362f-4d78-9379-a4e50a82a025 service neutron] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:23:11.033572 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1142/2307] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:23:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:23:11.034414 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e81c8a-362f-4d78-9379-a4e50a82a025 service neutron] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:23:11.044939 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e81c8a-362f-4d78-9379-a4e50a82a025 service neutron] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:23:11.051345 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfb5e8a-30b4-42b6-9cc8-6c802114b5d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:23:11.051942 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfb5e8a-30b4-42b6-9cc8-6c802114b5d0 admin admin] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:23:11.052254 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfb5e8a-30b4-42b6-9cc8-6c802114b5d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:23:11.052533 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfb5e8a-30b4-42b6-9cc8-6c802114b5d0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:23:11.053153 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfb5e8a-30b4-42b6-9cc8-6c802114b5d0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:23:11.055030 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfb5e8a-30b4-42b6-9cc8-6c802114b5d0 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:23:11.144135 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-da0ea919-f087-41b1-9cb1-9e8b48047d87 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '20075491d44d4be7a541cf92ecc722ee', 'request_id': 'req-da0ea919-f087-41b1-9cb1-9e8b48047d87'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:23:11.150291 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-da0ea919-f087-41b1-9cb1-9e8b48047d87 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '20075491d44d4be7a541cf92ecc722ee', 'request_id': 'req-da0ea919-f087-41b1-9cb1-9e8b48047d87'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:23:11.156077 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-da0ea919-f087-41b1-9cb1-9e8b48047d87 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '20075491d44d4be7a541cf92ecc722ee', 'request_id': 'req-da0ea919-f087-41b1-9cb1-9e8b48047d87'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:23:11.166203 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1143/2308] 10.222.0.55 () {60 vars in 1153 bytes} [Fri Sep 23 15:23:11 2022] POST /identity/v3/projects => generated 385 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:23:11.170694 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfb5e8a-30b4-42b6-9cc8-6c802114b5d0 admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:23:11.236316 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfb5e8a-30b4-42b6-9cc8-6c802114b5d0 admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:23:11.241067 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d67a95-a42f-4498-8ca6-a423dff2669b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:23:11.241681 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d67a95-a42f-4498-8ca6-a423dff2669b admin admin] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:23:11.241987 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d67a95-a42f-4498-8ca6-a423dff2669b admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:23:11.242233 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d67a95-a42f-4498-8ca6-a423dff2669b admin admin] PATH_INFO: `/v3/users` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:23:11.242858 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d67a95-a42f-4498-8ca6-a423dff2669b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:23:11.244892 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d67a95-a42f-4498-8ca6-a423dff2669b admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:23:11.300432 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-da0ea919-f087-41b1-9cb1-9e8b48047d87 admin admin] The token cache is being invalidate because project 20075491d44d4be7a541cf92ecc722ee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81994) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 15:23:11.304501 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-da0ea919-f087-41b1-9cb1-9e8b48047d87 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:23:11.304501 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-da0ea919-f087-41b1-9cb1-9e8b48047d87 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:23:11.304501 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1166/2309] 10.222.0.55 () {58 vars in 1233 bytes} [Fri Sep 23 15:23:10 2022] DELETE /identity/v3/projects/20075491d44d4be7a541cf92ecc722ee => generated 0 bytes in 538 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:23:11.310224 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64b7069-372f-40ba-8f1a-4015b84426d5 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:23:11.312457 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64b7069-372f-40ba-8f1a-4015b84426d5 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:23:11.320152 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64b7069-372f-40ba-8f1a-4015b84426d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:23:11.361633 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-c64b7069-372f-40ba-8f1a-4015b84426d5 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:23:11.362917 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-c64b7069-372f-40ba-8f1a-4015b84426d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:23:11.373727 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1167/2310] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:23:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:23:11.383047 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecdd52d-fe48-4d3c-a3aa-f723a2fb39bb None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:23:11.383446 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecdd52d-fe48-4d3c-a3aa-f723a2fb39bb None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:23:11.384274 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecdd52d-fe48-4d3c-a3aa-f723a2fb39bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:23:11.425501 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-4ecdd52d-fe48-4d3c-a3aa-f723a2fb39bb None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:23:11.427182 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-4ecdd52d-fe48-4d3c-a3aa-f723a2fb39bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:23:11.439697 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1144/2311] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Sep 23 15:23:11 2022] POST /identity/v3/users => generated 293 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 23 15:23:11.442971 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1168/2312] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:23:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:23:11.444024 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d67a95-a42f-4498-8ca6-a423dff2669b admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:23:11.444580 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecdd52d-fe48-4d3c-a3aa-f723a2fb39bb None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:23:11.457216 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecdd52d-fe48-4d3c-a3aa-f723a2fb39bb None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:23:11.461037 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-f3d67a95-a42f-4498-8ca6-a423dff2669b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:23:11.473136 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29de521c-a578-469d-8a0e-65366e5ab171 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:23:11.482509 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d67a95-a42f-4498-8ca6-a423dff2669b admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:23:11.482858 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29de521c-a578-469d-8a0e-65366e5ab171 admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:23:11.482858 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29de521c-a578-469d-8a0e-65366e5ab171 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:23:11.482858 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29de521c-a578-469d-8a0e-65366e5ab171 admin admin] PATH_INFO: `/v3/domains` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:23:11.482858 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29de521c-a578-469d-8a0e-65366e5ab171 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:23:11.482858 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29de521c-a578-469d-8a0e-65366e5ab171 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:23:11.482858 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29de521c-a578-469d-8a0e-65366e5ab171 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:23:11.483651 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a22264-aa2a-49d2-8b76-f0a4ff4054d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:23:11.485821 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a22264-aa2a-49d2-8b76-f0a4ff4054d8 admin admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:23:11.485821 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a22264-aa2a-49d2-8b76-f0a4ff4054d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:23:11.486045 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a22264-aa2a-49d2-8b76-f0a4ff4054d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:23:11.486526 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a22264-aa2a-49d2-8b76-f0a4ff4054d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:23:11.488858 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a22264-aa2a-49d2-8b76-f0a4ff4054d8 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:23:11.510222 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1145/2313] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Sep 23 15:23:11 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 23 15:23:11.513986 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a22264-aa2a-49d2-8b76-f0a4ff4054d8 admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:23:11.526227 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a22264-aa2a-49d2-8b76-f0a4ff4054d8 admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:23:11.528500 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1169/2314] 10.222.0.55 () {58 vars in 1153 bytes} [Fri Sep 23 15:23:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:23:11.533070 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29de521c-a578-469d-8a0e-65366e5ab171 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:23:11.540976 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27f1c59-c1aa-406f-8409-774eda63b73a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:23:11.552807 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27f1c59-c1aa-406f-8409-774eda63b73a admin admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:23:11.554078 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27f1c59-c1aa-406f-8409-774eda63b73a admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:23:11.554362 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27f1c59-c1aa-406f-8409-774eda63b73a admin admin] PATH_INFO: `/v3/roles` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:23:11.558538 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27f1c59-c1aa-406f-8409-774eda63b73a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:23:11.563709 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27f1c59-c1aa-406f-8409-774eda63b73a admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:23:11.570592 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29de521c-a578-469d-8a0e-65366e5ab171 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:23:11.590850 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e19857e-ec98-436b-bf5c-1fc20f4fc013 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:23:11.592192 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e19857e-ec98-436b-bf5c-1fc20f4fc013 admin admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:23:11.593031 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e19857e-ec98-436b-bf5c-1fc20f4fc013 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:23:11.593236 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e19857e-ec98-436b-bf5c-1fc20f4fc013 admin admin] PATH_INFO: `/v3/projects` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:23:11.594137 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e19857e-ec98-436b-bf5c-1fc20f4fc013 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:23:11.604814 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e19857e-ec98-436b-bf5c-1fc20f4fc013 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:23:11.609936 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1146/2315] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Sep 23 15:23:11 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:23:11.618804 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27f1c59-c1aa-406f-8409-774eda63b73a admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:23:11.642200 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27f1c59-c1aa-406f-8409-774eda63b73a admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:23:11.654690 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f34519b-4556-4a6e-9de3-867fad788d9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:23:11.655412 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f34519b-4556-4a6e-9de3-867fad788d9c admin admin] REQUEST_METHOD: `PUT` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:23:11.655695 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f34519b-4556-4a6e-9de3-867fad788d9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:23:11.656230 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f34519b-4556-4a6e-9de3-867fad788d9c admin admin] PATH_INFO: `/v3/projects/86aa75dc0b724749ab4cd37b65666154/users/668fbf0ee9f444ac84b075a2ec397af2/roles/af257be2dee54d6b942f58650196582c` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:23:11.676454 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f34519b-4556-4a6e-9de3-867fad788d9c admin admin] RBAC: Authorizing `identity:create_grant(project_id=86aa75dc0b724749ab4cd37b65666154, user_id=668fbf0ee9f444ac84b075a2ec397af2, role_id=af257be2dee54d6b942f58650196582c)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:23:11.680192 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f34519b-4556-4a6e-9de3-867fad788d9c admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:23:11.758714 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1170/2316] 10.222.0.55 () {60 vars in 1153 bytes} [Fri Sep 23 15:23:11 2022] POST /identity/v3/projects => generated 375 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:23:11.759724 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e19857e-ec98-436b-bf5c-1fc20f4fc013 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:23:11.795283 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1147/2317] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Sep 23 15:23:11 2022] PUT /identity/v3/projects/86aa75dc0b724749ab4cd37b65666154/users/668fbf0ee9f444ac84b075a2ec397af2/roles/af257be2dee54d6b942f58650196582c => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:23:11.806995 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0b09a4-924f-4482-afe3-acf986411d09 None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:23:11.807462 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0b09a4-924f-4482-afe3-acf986411d09 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:23:11.807803 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0b09a4-924f-4482-afe3-acf986411d09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:23:11.863251 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e19857e-ec98-436b-bf5c-1fc20f4fc013 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:23:11.871349 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-bc0b09a4-924f-4482-afe3-acf986411d09 None None] MFA Rules not processed for user `668fbf0ee9f444ac84b075a2ec397af2`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:23:11.873269 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-bc0b09a4-924f-4482-afe3-acf986411d09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:23:11.959538 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ec929e-9f8c-4c01-88d8-6cc0249f668b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:23:11.960703 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ec929e-9f8c-4c01-88d8-6cc0249f668b admin admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:23:11.961272 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ec929e-9f8c-4c01-88d8-6cc0249f668b admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:23:11.962081 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ec929e-9f8c-4c01-88d8-6cc0249f668b admin admin] PATH_INFO: `/v3/users` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:23:11.962889 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ec929e-9f8c-4c01-88d8-6cc0249f668b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:23:11.964861 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ec929e-9f8c-4c01-88d8-6cc0249f668b admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:23:11.986831 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1148/2318] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:23:11 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 15:23:11.989671 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0b09a4-924f-4482-afe3-acf986411d09 None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:23:11.993718 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-bc0b09a4-924f-4482-afe3-acf986411d09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:23:12.013674 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0b09a4-924f-4482-afe3-acf986411d09 None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:23:12.019461 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fc3352-c84b-4f35-ab2f-e470ae8dbab6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:23:12.020213 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fc3352-c84b-4f35-ab2f-e470ae8dbab6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:23:12.020496 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fc3352-c84b-4f35-ab2f-e470ae8dbab6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:23:12.020767 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fc3352-c84b-4f35-ab2f-e470ae8dbab6 admin admin] PATH_INFO: `/v3/projects/da5d992d91c34ac99c928ec3fedaf320` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:23:12.021898 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fc3352-c84b-4f35-ab2f-e470ae8dbab6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=da5d992d91c34ac99c928ec3fedaf320)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:23:12.023989 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fc3352-c84b-4f35-ab2f-e470ae8dbab6 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:23:12.148698 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:23:12.154761 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1171/2319] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Sep 23 15:23:11 2022] POST /identity/v3/users => generated 288 bytes in 392 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:23:12.154761 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ec929e-9f8c-4c01-88d8-6cc0249f668b admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:23:12.156191 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-88ec929e-9f8c-4c01-88d8-6cc0249f668b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:23:12.178386 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-97fc3352-c84b-4f35-ab2f-e470ae8dbab6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'da5d992d91c34ac99c928ec3fedaf320', 'request_id': 'req-97fc3352-c84b-4f35-ab2f-e470ae8dbab6'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:23:12.185680 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-97fc3352-c84b-4f35-ab2f-e470ae8dbab6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'da5d992d91c34ac99c928ec3fedaf320', 'request_id': 'req-97fc3352-c84b-4f35-ab2f-e470ae8dbab6'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:23:12.227094 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ec929e-9f8c-4c01-88d8-6cc0249f668b admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:23:12.280990 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabd50dd-15d2-4591-849d-984c80d01e7d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feca6278be904ef8b0dc580a67e477f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:23:12.281632 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabd50dd-15d2-4591-849d-984c80d01e7d service cinder] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:23:12.282031 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabd50dd-15d2-4591-849d-984c80d01e7d service cinder] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:23:12.282313 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabd50dd-15d2-4591-849d-984c80d01e7d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:23:12.349079 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabd50dd-15d2-4591-849d-984c80d01e7d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:23:12.351303 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabd50dd-15d2-4591-849d-984c80d01e7d service cinder] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:23:12.362477 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1172/2320] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:23:12 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:23:12.363771 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabd50dd-15d2-4591-849d-984c80d01e7d service cinder] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:23:12.393342 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-97fc3352-c84b-4f35-ab2f-e470ae8dbab6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'da5d992d91c34ac99c928ec3fedaf320', 'request_id': 'req-97fc3352-c84b-4f35-ab2f-e470ae8dbab6'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:23:12.416349 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-97fc3352-c84b-4f35-ab2f-e470ae8dbab6 admin admin] The token cache is being invalidate because project da5d992d91c34ac99c928ec3fedaf320 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81993) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 15:23:12.417031 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-97fc3352-c84b-4f35-ab2f-e470ae8dbab6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:23:12.418138 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-97fc3352-c84b-4f35-ab2f-e470ae8dbab6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:23:12.421871 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1149/2321] 10.222.0.55 () {58 vars in 1233 bytes} [Fri Sep 23 15:23:11 2022] DELETE /identity/v3/projects/da5d992d91c34ac99c928ec3fedaf320 => generated 0 bytes in 434 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:23:12.426200 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fc3352-c84b-4f35-ab2f-e470ae8dbab6 admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:23:12.427638 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-97fc3352-c84b-4f35-ab2f-e470ae8dbab6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:23:12.470556 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabd50dd-15d2-4591-849d-984c80d01e7d service cinder] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:23:12.474989 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fc3352-c84b-4f35-ab2f-e470ae8dbab6 admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:23:12.480542 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2608bd4-1897-4e9f-98a2-0de8f9acae21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:23:12.481178 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2608bd4-1897-4e9f-98a2-0de8f9acae21 admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:23:12.482029 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2608bd4-1897-4e9f-98a2-0de8f9acae21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:23:12.482400 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2608bd4-1897-4e9f-98a2-0de8f9acae21 admin admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:23:12.483374 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2608bd4-1897-4e9f-98a2-0de8f9acae21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:23:12.485191 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2608bd4-1897-4e9f-98a2-0de8f9acae21 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:23:12.493792 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b834e57-ad49-4a47-8ab5-15a05a4ffc5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:23:12.494452 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b834e57-ad49-4a47-8ab5-15a05a4ffc5e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:23:12.494731 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b834e57-ad49-4a47-8ab5-15a05a4ffc5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:23:12.495201 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b834e57-ad49-4a47-8ab5-15a05a4ffc5e admin admin] PATH_INFO: `/v3/users/d9538a9e1d8741d3a1bb1dfd3383b47d` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:23:12.497574 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b834e57-ad49-4a47-8ab5-15a05a4ffc5e admin admin] RBAC: Authorizing `identity:delete_user(user_id=d9538a9e1d8741d3a1bb1dfd3383b47d)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:23:12.500745 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b834e57-ad49-4a47-8ab5-15a05a4ffc5e admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:23:12.512248 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1173/2322] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Sep 23 15:23:12 2022] GET /identity/v3/roles => generated 1420 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:23:12.515246 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2608bd4-1897-4e9f-98a2-0de8f9acae21 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:23:12.519314 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-e2608bd4-1897-4e9f-98a2-0de8f9acae21 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:23:12.547103 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2608bd4-1897-4e9f-98a2-0de8f9acae21 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:23:12.558031 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd21cf2c-aaed-4cc6-85fc-d394be8a0128 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:23:12.559816 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd21cf2c-aaed-4cc6-85fc-d394be8a0128 admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:23:12.560466 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd21cf2c-aaed-4cc6-85fc-d394be8a0128 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:23:12.560868 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd21cf2c-aaed-4cc6-85fc-d394be8a0128 admin admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:23:12.561639 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd21cf2c-aaed-4cc6-85fc-d394be8a0128 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:23:12.565548 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd21cf2c-aaed-4cc6-85fc-d394be8a0128 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:23:12.580403 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1174/2323] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Sep 23 15:23:12 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:23:12.584642 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd21cf2c-aaed-4cc6-85fc-d394be8a0128 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:23:12.596879 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd21cf2c-aaed-4cc6-85fc-d394be8a0128 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:23:12.602318 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a893d6-8388-40b8-b1b2-052aea8357fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:23:12.603071 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a893d6-8388-40b8-b1b2-052aea8357fe admin admin] REQUEST_METHOD: `PUT` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:23:12.603532 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a893d6-8388-40b8-b1b2-052aea8357fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:23:12.603836 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a893d6-8388-40b8-b1b2-052aea8357fe admin admin] PATH_INFO: `/v3/projects/10fdbae91862410ebdcdff525598875e/users/42b46c6aac51477da1ead5348445e2ea/roles/af257be2dee54d6b942f58650196582c` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:23:12.620565 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a893d6-8388-40b8-b1b2-052aea8357fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=10fdbae91862410ebdcdff525598875e, user_id=42b46c6aac51477da1ead5348445e2ea, role_id=af257be2dee54d6b942f58650196582c)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:23:12.623852 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a893d6-8388-40b8-b1b2-052aea8357fe admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:23:12.768128 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1175/2324] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Sep 23 15:23:12 2022] PUT /identity/v3/projects/10fdbae91862410ebdcdff525598875e/users/42b46c6aac51477da1ead5348445e2ea/roles/af257be2dee54d6b942f58650196582c => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 15:23:12.779152 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2720b4b-92b6-4cf0-8d56-2c03d33fc3f3 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:23:12.779458 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2720b4b-92b6-4cf0-8d56-2c03d33fc3f3 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:23:12.779718 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2720b4b-92b6-4cf0-8d56-2c03d33fc3f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:23:12.804954 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-0b834e57-ad49-4a47-8ab5-15a05a4ffc5e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd9538a9e1d8741d3a1bb1dfd3383b47d'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:23:12.813487 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-f2720b4b-92b6-4cf0-8d56-2c03d33fc3f3 None None] MFA Rules not processed for user `42b46c6aac51477da1ead5348445e2ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:23:12.814916 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-f2720b4b-92b6-4cf0-8d56-2c03d33fc3f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:23:12.818913 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-0b834e57-ad49-4a47-8ab5-15a05a4ffc5e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd9538a9e1d8741d3a1bb1dfd3383b47d'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:23:12.825196 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:23:12.826523 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-0b834e57-ad49-4a47-8ab5-15a05a4ffc5e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd9538a9e1d8741d3a1bb1dfd3383b47d'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:23:12.898768 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1176/2325] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:23:12 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 15:23:12.902199 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2720b4b-92b6-4cf0-8d56-2c03d33fc3f3 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:23:12.909554 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-0b834e57-ad49-4a47-8ab5-15a05a4ffc5e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd9538a9e1d8741d3a1bb1dfd3383b47d'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:23:12.910498 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-0b834e57-ad49-4a47-8ab5-15a05a4ffc5e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd9538a9e1d8741d3a1bb1dfd3383b47d'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:23:12.924406 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1150/2326] 10.222.0.55 () {58 vars in 1224 bytes} [Fri Sep 23 15:23:12 2022] DELETE /identity/v3/users/d9538a9e1d8741d3a1bb1dfd3383b47d => generated 0 bytes in 498 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:23:13.004984 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2720b4b-92b6-4cf0-8d56-2c03d33fc3f3 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:23:13.089421 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b05e1d-2b16-4d62-89ac-0c969598e2f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:23:13.090056 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b05e1d-2b16-4d62-89ac-0c969598e2f9 admin admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:23:13.090344 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b05e1d-2b16-4d62-89ac-0c969598e2f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:23:13.090669 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b05e1d-2b16-4d62-89ac-0c969598e2f9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:23:13.092373 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b05e1d-2b16-4d62-89ac-0c969598e2f9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:23:13.094432 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b05e1d-2b16-4d62-89ac-0c969598e2f9 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:23:13.258115 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1177/2327] 10.222.0.55 () {60 vars in 1153 bytes} [Fri Sep 23 15:23:12 2022] POST /identity/v3/projects => generated 375 bytes in 356 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 23 15:23:13.262315 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b834e57-ad49-4a47-8ab5-15a05a4ffc5e admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:23:13.265268 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-0b834e57-ad49-4a47-8ab5-15a05a4ffc5e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:23:13.278870 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b834e57-ad49-4a47-8ab5-15a05a4ffc5e admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:23:13.337186 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcb76ad-5f74-49a8-af0f-4532fc2bdf8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:23:13.337836 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcb76ad-5f74-49a8-af0f-4532fc2bdf8d admin admin] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:23:13.338536 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcb76ad-5f74-49a8-af0f-4532fc2bdf8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:23:13.338536 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcb76ad-5f74-49a8-af0f-4532fc2bdf8d admin admin] PATH_INFO: `/v3/users` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:23:13.339047 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcb76ad-5f74-49a8-af0f-4532fc2bdf8d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:23:13.341066 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcb76ad-5f74-49a8-af0f-4532fc2bdf8d admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:23:13.542229 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1151/2328] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Sep 23 15:23:13 2022] POST /identity/v3/users => generated 294 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:23:13.543676 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b05e1d-2b16-4d62-89ac-0c969598e2f9 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:23:13.562686 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b05e1d-2b16-4d62-89ac-0c969598e2f9 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:23:13.568377 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82174bf2-77c8-49a3-b5c9-31a3f5bfa0ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:23:13.569038 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82174bf2-77c8-49a3-b5c9-31a3f5bfa0ce admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:23:13.569325 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82174bf2-77c8-49a3-b5c9-31a3f5bfa0ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:23:13.569607 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82174bf2-77c8-49a3-b5c9-31a3f5bfa0ce admin admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:23:13.570272 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82174bf2-77c8-49a3-b5c9-31a3f5bfa0ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:23:13.571995 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82174bf2-77c8-49a3-b5c9-31a3f5bfa0ce admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:23:13.587870 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1178/2329] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Sep 23 15:23:13 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:23:13.592036 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcb76ad-5f74-49a8-af0f-4532fc2bdf8d admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:23:13.603664 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcb76ad-5f74-49a8-af0f-4532fc2bdf8d admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:23:13.609269 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f526d61-85d4-436f-9ed2-53e187ab19e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:23:13.610000 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f526d61-85d4-436f-9ed2-53e187ab19e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:23:13.610436 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f526d61-85d4-436f-9ed2-53e187ab19e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:23:13.610775 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f526d61-85d4-436f-9ed2-53e187ab19e6 admin admin] PATH_INFO: `/v3/projects/6450c1191bf148279d65ed2c73baba58/users/07c58b7fda4146ef969b0c0d953ea95e/roles/8e2919e79df040e8926f21607818b58c` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:23:13.628746 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f526d61-85d4-436f-9ed2-53e187ab19e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6450c1191bf148279d65ed2c73baba58, user_id=07c58b7fda4146ef969b0c0d953ea95e, role_id=8e2919e79df040e8926f21607818b58c)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:23:13.631637 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f526d61-85d4-436f-9ed2-53e187ab19e6 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:23:13.652357 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82174bf2-77c8-49a3-b5c9-31a3f5bfa0ce admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:23:13.654535 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-82174bf2-77c8-49a3-b5c9-31a3f5bfa0ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:23:13.675755 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82174bf2-77c8-49a3-b5c9-31a3f5bfa0ce admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:23:13.681330 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d74ce04-5476-46e2-925f-520ffe234ce3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:23:13.682363 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d74ce04-5476-46e2-925f-520ffe234ce3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:23:13.682749 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d74ce04-5476-46e2-925f-520ffe234ce3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:23:13.683182 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d74ce04-5476-46e2-925f-520ffe234ce3 admin admin] PATH_INFO: `/v3/projects/435f82ccb24c40e6b7dedc2e5cb15174` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:23:13.685339 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d74ce04-5476-46e2-925f-520ffe234ce3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=435f82ccb24c40e6b7dedc2e5cb15174)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:23:13.687642 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d74ce04-5476-46e2-925f-520ffe234ce3 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:23:13.727211 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1152/2330] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Sep 23 15:23:13 2022] PUT /identity/v3/projects/6450c1191bf148279d65ed2c73baba58/users/07c58b7fda4146ef969b0c0d953ea95e/roles/8e2919e79df040e8926f21607818b58c => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:23:13.730676 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f526d61-85d4-436f-9ed2-53e187ab19e6 admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:23:13.800220 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f526d61-85d4-436f-9ed2-53e187ab19e6 admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:23:13.813870 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5cb5f6-8f70-4fcc-bd79-6282d869d9d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:23:13.814593 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5cb5f6-8f70-4fcc-bd79-6282d869d9d9 admin admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:23:13.814888 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5cb5f6-8f70-4fcc-bd79-6282d869d9d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:23:13.817306 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5cb5f6-8f70-4fcc-bd79-6282d869d9d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:23:13.817306 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5cb5f6-8f70-4fcc-bd79-6282d869d9d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:23:13.817745 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5cb5f6-8f70-4fcc-bd79-6282d869d9d9 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:23:13.832083 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-3d74ce04-5476-46e2-925f-520ffe234ce3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '435f82ccb24c40e6b7dedc2e5cb15174', 'request_id': 'req-3d74ce04-5476-46e2-925f-520ffe234ce3'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:23:13.832361 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1153/2331] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Sep 23 15:23:13 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:23:13.835887 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5cb5f6-8f70-4fcc-bd79-6282d869d9d9 admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:23:13.843727 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-3d74ce04-5476-46e2-925f-520ffe234ce3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '435f82ccb24c40e6b7dedc2e5cb15174', 'request_id': 'req-3d74ce04-5476-46e2-925f-520ffe234ce3'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:23:13.846755 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5cb5f6-8f70-4fcc-bd79-6282d869d9d9 admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:23:13.850442 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-3d74ce04-5476-46e2-925f-520ffe234ce3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '435f82ccb24c40e6b7dedc2e5cb15174', 'request_id': 'req-3d74ce04-5476-46e2-925f-520ffe234ce3'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:23:13.918118 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d339e3ae-d4d8-493d-a02c-ee61c662bae3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:23:13.919436 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d339e3ae-d4d8-493d-a02c-ee61c662bae3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:23:13.919833 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d339e3ae-d4d8-493d-a02c-ee61c662bae3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:23:13.921027 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d339e3ae-d4d8-493d-a02c-ee61c662bae3 admin admin] PATH_INFO: `/v3/projects/6450c1191bf148279d65ed2c73baba58/users/07c58b7fda4146ef969b0c0d953ea95e/roles/8e2919e79df040e8926f21607818b58c` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:23:13.924507 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d339e3ae-d4d8-493d-a02c-ee61c662bae3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6450c1191bf148279d65ed2c73baba58, user_id=07c58b7fda4146ef969b0c0d953ea95e, role_id=8e2919e79df040e8926f21607818b58c)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:23:13.938232 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d339e3ae-d4d8-493d-a02c-ee61c662bae3 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:23:13.955440 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1154/2332] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Sep 23 15:23:13 2022] PUT /identity/v3/projects/6450c1191bf148279d65ed2c73baba58/users/07c58b7fda4146ef969b0c0d953ea95e/roles/8e2919e79df040e8926f21607818b58c => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 15:23:13.972388 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febbb81f-1982-487e-bb9d-2bfa47f2af17 None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:23:13.972673 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febbb81f-1982-487e-bb9d-2bfa47f2af17 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:23:13.972948 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febbb81f-1982-487e-bb9d-2bfa47f2af17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:23:14.008096 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-febbb81f-1982-487e-bb9d-2bfa47f2af17 None None] MFA Rules not processed for user `07c58b7fda4146ef969b0c0d953ea95e`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:23:14.009320 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-febbb81f-1982-487e-bb9d-2bfa47f2af17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:23:14.011661 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-3d74ce04-5476-46e2-925f-520ffe234ce3 admin admin] The token cache is being invalidate because project 435f82ccb24c40e6b7dedc2e5cb15174 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81994) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 15:23:14.012430 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-3d74ce04-5476-46e2-925f-520ffe234ce3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:23:14.014016 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-3d74ce04-5476-46e2-925f-520ffe234ce3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:23:14.016026 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1179/2333] 10.222.0.55 () {58 vars in 1233 bytes} [Fri Sep 23 15:23:13 2022] DELETE /identity/v3/projects/435f82ccb24c40e6b7dedc2e5cb15174 => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 15:23:14.025534 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d74ce04-5476-46e2-925f-520ffe234ce3 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:23:14.026950 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-3d74ce04-5476-46e2-925f-520ffe234ce3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:23:14.042085 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d74ce04-5476-46e2-925f-520ffe234ce3 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:23:14.122886 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407b19e9-39cd-43fb-a946-3ab43dc1ae85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:23:14.123701 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407b19e9-39cd-43fb-a946-3ab43dc1ae85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:23:14.124102 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407b19e9-39cd-43fb-a946-3ab43dc1ae85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:23:14.124387 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407b19e9-39cd-43fb-a946-3ab43dc1ae85 admin admin] PATH_INFO: `/v3/users/b645586fc8d140cc969cff59b63c44b6` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:23:14.125588 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407b19e9-39cd-43fb-a946-3ab43dc1ae85 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b645586fc8d140cc969cff59b63c44b6)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:23:14.127511 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1155/2334] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:23:13 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:23:14.129383 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407b19e9-39cd-43fb-a946-3ab43dc1ae85 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:23:14.147670 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febbb81f-1982-487e-bb9d-2bfa47f2af17 None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:23:14.151370 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-febbb81f-1982-487e-bb9d-2bfa47f2af17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:23:14.283681 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febbb81f-1982-487e-bb9d-2bfa47f2af17 None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:23:14.298075 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e99a3a6-7e71-47c5-990e-0c486dff9966 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feca6278be904ef8b0dc580a67e477f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:23:14.298687 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e99a3a6-7e71-47c5-990e-0c486dff9966 service cinder] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:23:14.298977 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e99a3a6-7e71-47c5-990e-0c486dff9966 service cinder] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:23:14.299238 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e99a3a6-7e71-47c5-990e-0c486dff9966 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:23:14.305969 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-8e99a3a6-7e71-47c5-990e-0c486dff9966 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:23:14.386753 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e99a3a6-7e71-47c5-990e-0c486dff9966 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:23:14.389487 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e99a3a6-7e71-47c5-990e-0c486dff9966 service cinder] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:23:14.408336 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1156/2335] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:23:14 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 267 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:23:14.477030 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-407b19e9-39cd-43fb-a946-3ab43dc1ae85 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b645586fc8d140cc969cff59b63c44b6'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:23:14.499126 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-407b19e9-39cd-43fb-a946-3ab43dc1ae85 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b645586fc8d140cc969cff59b63c44b6'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:23:14.514402 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-407b19e9-39cd-43fb-a946-3ab43dc1ae85 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b645586fc8d140cc969cff59b63c44b6'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:23:14.650660 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-407b19e9-39cd-43fb-a946-3ab43dc1ae85 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b645586fc8d140cc969cff59b63c44b6'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:23:14.651337 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-407b19e9-39cd-43fb-a946-3ab43dc1ae85 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b645586fc8d140cc969cff59b63c44b6'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:23:14.658938 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1180/2336] 10.222.0.55 () {58 vars in 1224 bytes} [Fri Sep 23 15:23:14 2022] DELETE /identity/v3/users/b645586fc8d140cc969cff59b63c44b6 => generated 0 bytes in 634 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 15:23:14.822236 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:23:15.291601 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e99a3a6-7e71-47c5-990e-0c486dff9966 service cinder] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:23:15.324195 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e99a3a6-7e71-47c5-990e-0c486dff9966 service cinder] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:23:15.411552 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc94564-3897-46b7-86cd-432bc2dcfb71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:23:15.412257 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc94564-3897-46b7-86cd-432bc2dcfb71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:23:15.412537 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc94564-3897-46b7-86cd-432bc2dcfb71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:23:15.412802 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc94564-3897-46b7-86cd-432bc2dcfb71 admin admin] PATH_INFO: `/v3/projects/6dc7f4d2c9cc45d780ef4d56d1c1964e` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:23:15.413916 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc94564-3897-46b7-86cd-432bc2dcfb71 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6dc7f4d2c9cc45d780ef4d56d1c1964e)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:23:15.415933 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc94564-3897-46b7-86cd-432bc2dcfb71 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:23:15.571842 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-5dc94564-3897-46b7-86cd-432bc2dcfb71 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6dc7f4d2c9cc45d780ef4d56d1c1964e', 'request_id': 'req-5dc94564-3897-46b7-86cd-432bc2dcfb71'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:23:15.579279 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-5dc94564-3897-46b7-86cd-432bc2dcfb71 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6dc7f4d2c9cc45d780ef4d56d1c1964e', 'request_id': 'req-5dc94564-3897-46b7-86cd-432bc2dcfb71'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:23:15.741680 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-5dc94564-3897-46b7-86cd-432bc2dcfb71 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6dc7f4d2c9cc45d780ef4d56d1c1964e', 'request_id': 'req-5dc94564-3897-46b7-86cd-432bc2dcfb71'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:23:15.761832 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-5dc94564-3897-46b7-86cd-432bc2dcfb71 admin admin] The token cache is being invalidate because project 6dc7f4d2c9cc45d780ef4d56d1c1964e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81993) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 15:23:15.762136 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-5dc94564-3897-46b7-86cd-432bc2dcfb71 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:23:15.763591 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-5dc94564-3897-46b7-86cd-432bc2dcfb71 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:23:15.765543 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1157/2337] 10.222.0.55 () {58 vars in 1233 bytes} [Fri Sep 23 15:23:15 2022] DELETE /identity/v3/projects/6dc7f4d2c9cc45d780ef4d56d1c1964e => generated 0 bytes in 475 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:23:15.771163 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e8bb57-331e-4ff1-89a1-eb501ae5a184 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:23:15.771701 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e8bb57-331e-4ff1-89a1-eb501ae5a184 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:23:15.771877 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e8bb57-331e-4ff1-89a1-eb501ae5a184 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:23:15.794526 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-90e8bb57-331e-4ff1-89a1-eb501ae5a184 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:23:15.795922 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-90e8bb57-331e-4ff1-89a1-eb501ae5a184 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:23:15.861095 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1181/2338] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:23:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:23:15.869434 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d1cf77-5141-4b33-be5c-50bfd1f51f5a None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:23:15.870603 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d1cf77-5141-4b33-be5c-50bfd1f51f5a None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:23:15.870603 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d1cf77-5141-4b33-be5c-50bfd1f51f5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:23:15.894540 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-26d1cf77-5141-4b33-be5c-50bfd1f51f5a None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:23:15.895757 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-26d1cf77-5141-4b33-be5c-50bfd1f51f5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:23:15.904763 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1158/2339] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:23:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 15:23:15.914172 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e8bb57-331e-4ff1-89a1-eb501ae5a184 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:23:15.930865 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e8bb57-331e-4ff1-89a1-eb501ae5a184 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:23:15.930865 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c776d7-0604-44fb-a4fe-2d1b968911eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:23:15.931055 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c776d7-0604-44fb-a4fe-2d1b968911eb admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:23:15.931328 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c776d7-0604-44fb-a4fe-2d1b968911eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:23:15.931557 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c776d7-0604-44fb-a4fe-2d1b968911eb admin admin] PATH_INFO: `/v3/domains` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:23:15.932242 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c776d7-0604-44fb-a4fe-2d1b968911eb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:23:15.932555 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c776d7-0604-44fb-a4fe-2d1b968911eb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:23:15.934152 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c776d7-0604-44fb-a4fe-2d1b968911eb admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:23:15.947516 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1182/2340] 10.222.0.55 () {58 vars in 1153 bytes} [Fri Sep 23 15:23:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:23:15.952442 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d1cf77-5141-4b33-be5c-50bfd1f51f5a None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:23:15.962294 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d1cf77-5141-4b33-be5c-50bfd1f51f5a None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:23:15.967515 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196bcdce-705a-4c06-b34a-65f9e28d362b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:23:15.968300 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196bcdce-705a-4c06-b34a-65f9e28d362b admin admin] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:23:15.968596 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196bcdce-705a-4c06-b34a-65f9e28d362b admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:23:15.968973 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196bcdce-705a-4c06-b34a-65f9e28d362b admin admin] PATH_INFO: `/v3/projects` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:23:15.969607 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196bcdce-705a-4c06-b34a-65f9e28d362b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:23:15.971501 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196bcdce-705a-4c06-b34a-65f9e28d362b admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:23:16.138602 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1159/2341] 10.222.0.55 () {60 vars in 1153 bytes} [Fri Sep 23 15:23:15 2022] POST /identity/v3/projects => generated 373 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:23:16.143192 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c776d7-0604-44fb-a4fe-2d1b968911eb admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:23:16.154198 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c776d7-0604-44fb-a4fe-2d1b968911eb admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:23:16.219785 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397e7b50-6995-4a2c-8a39-e92d6ec78e3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:23:16.220466 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397e7b50-6995-4a2c-8a39-e92d6ec78e3f admin admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:23:16.220754 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397e7b50-6995-4a2c-8a39-e92d6ec78e3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:23:16.221033 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397e7b50-6995-4a2c-8a39-e92d6ec78e3f admin admin] PATH_INFO: `/v3/users` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:23:16.221679 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397e7b50-6995-4a2c-8a39-e92d6ec78e3f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:23:16.223610 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397e7b50-6995-4a2c-8a39-e92d6ec78e3f admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:23:16.412637 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1183/2342] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Sep 23 15:23:16 2022] POST /identity/v3/users => generated 287 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 23 15:23:16.413862 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979e85e2-63a4-4ade-9037-751631a1832b None None] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:23:16.414304 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979e85e2-63a4-4ade-9037-751631a1832b None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:23:16.414492 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979e85e2-63a4-4ade-9037-751631a1832b None None] PATH_INFO: `/` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:23:16.415459 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397e7b50-6995-4a2c-8a39-e92d6ec78e3f admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:23:16.416163 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1160/2343] 10.222.0.55 () {54 vars in 921 bytes} [Fri Sep 23 15:23:16 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 23 15:23:16.428322 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ebd3f4-9a86-4c8d-b032-8c0ceff9f038 None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:23:16.428322 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ebd3f4-9a86-4c8d-b032-8c0ceff9f038 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:23:16.428322 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ebd3f4-9a86-4c8d-b032-8c0ceff9f038 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:23:16.437951 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397e7b50-6995-4a2c-8a39-e92d6ec78e3f admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:23:16.443810 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfce25f-1955-48b5-b558-775b9f7787fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:23:16.444677 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfce25f-1955-48b5-b558-775b9f7787fd admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:23:16.451200 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfce25f-1955-48b5-b558-775b9f7787fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:23:16.451200 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfce25f-1955-48b5-b558-775b9f7787fd admin admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:23:16.451200 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfce25f-1955-48b5-b558-775b9f7787fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:23:16.451586 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-f6ebd3f4-9a86-4c8d-b032-8c0ceff9f038 None None] MFA Rules not processed for user `b3c6c7a1325a47be924268e1aa1a7a18`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:23:16.451681 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfce25f-1955-48b5-b558-775b9f7787fd admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:23:16.451758 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-f6ebd3f4-9a86-4c8d-b032-8c0ceff9f038 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:23:16.475539 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1184/2344] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Sep 23 15:23:16 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:23:16.479587 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfce25f-1955-48b5-b558-775b9f7787fd admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:23:16.495193 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfce25f-1955-48b5-b558-775b9f7787fd admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:23:16.501932 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ce525a-6c4f-4209-8215-9f236d2eb369 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:23:16.502559 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ce525a-6c4f-4209-8215-9f236d2eb369 admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:23:16.502858 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ce525a-6c4f-4209-8215-9f236d2eb369 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:23:16.505140 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ce525a-6c4f-4209-8215-9f236d2eb369 admin admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:23:16.505140 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ce525a-6c4f-4209-8215-9f236d2eb369 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:23:16.507337 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ce525a-6c4f-4209-8215-9f236d2eb369 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:23:16.520805 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1185/2345] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Sep 23 15:23:16 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:23:16.524981 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ce525a-6c4f-4209-8215-9f236d2eb369 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:23:16.536503 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ce525a-6c4f-4209-8215-9f236d2eb369 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:23:16.541384 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d729c1-1ebb-468e-ac1c-6d4a3c834c3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:23:16.542425 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d729c1-1ebb-468e-ac1c-6d4a3c834c3d admin admin] REQUEST_METHOD: `PUT` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:23:16.542784 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d729c1-1ebb-468e-ac1c-6d4a3c834c3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:23:16.543148 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d729c1-1ebb-468e-ac1c-6d4a3c834c3d admin admin] PATH_INFO: `/v3/projects/4073032ebdc54b3b95e818cf3c5a3878/users/260ef126a1e74fbe8bd23bc977bf97ab/roles/af257be2dee54d6b942f58650196582c` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:23:16.548580 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1161/2346] 10.222.0.55 () {58 vars in 1018 bytes} [Fri Sep 23 15:23:16 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:23:16.562620 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ebd3f4-9a86-4c8d-b032-8c0ceff9f038 None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:23:16.564139 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-f6ebd3f4-9a86-4c8d-b032-8c0ceff9f038 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:23:16.567577 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d729c1-1ebb-468e-ac1c-6d4a3c834c3d admin admin] RBAC: Authorizing `identity:create_grant(project_id=4073032ebdc54b3b95e818cf3c5a3878, user_id=260ef126a1e74fbe8bd23bc977bf97ab, role_id=af257be2dee54d6b942f58650196582c)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:23:16.570293 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d729c1-1ebb-468e-ac1c-6d4a3c834c3d admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:23:16.626028 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ebd3f4-9a86-4c8d-b032-8c0ceff9f038 None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:23:16.629789 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dcb0d2-b487-4877-92fa-aae2237cb8b8 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51c39f1d05634589a88c0b6477bd014e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:23:16.630380 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dcb0d2-b487-4877-92fa-aae2237cb8b8 service glance] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:23:16.630911 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dcb0d2-b487-4877-92fa-aae2237cb8b8 service glance] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:23:16.631197 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dcb0d2-b487-4877-92fa-aae2237cb8b8 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:23:16.642218 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0dcb0d2-b487-4877-92fa-aae2237cb8b8 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:23:16.643640 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0dcb0d2-b487-4877-92fa-aae2237cb8b8 service glance] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:23:16.654578 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1162/2347] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:23:16 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:23:16.722745 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1186/2348] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Sep 23 15:23:16 2022] PUT /identity/v3/projects/4073032ebdc54b3b95e818cf3c5a3878/users/260ef126a1e74fbe8bd23bc977bf97ab/roles/af257be2dee54d6b942f58650196582c => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 15:23:16.798361 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6828ce-c737-42bb-afeb-e2b7d803835e None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:23:16.798663 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6828ce-c737-42bb-afeb-e2b7d803835e None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:23:16.798944 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6828ce-c737-42bb-afeb-e2b7d803835e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:23:16.832544 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-0e6828ce-c737-42bb-afeb-e2b7d803835e None None] MFA Rules not processed for user `260ef126a1e74fbe8bd23bc977bf97ab`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:23:16.833717 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-0e6828ce-c737-42bb-afeb-e2b7d803835e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:23:16.906911 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1163/2349] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:23:16 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:23:16.916693 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d729c1-1ebb-468e-ac1c-6d4a3c834c3d admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:23:16.918487 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-a3d729c1-1ebb-468e-ac1c-6d4a3c834c3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:23:16.959280 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6828ce-c737-42bb-afeb-e2b7d803835e None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:23:16.960722 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-0e6828ce-c737-42bb-afeb-e2b7d803835e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:23:16.986201 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d729c1-1ebb-468e-ac1c-6d4a3c834c3d admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:23:16.990086 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50ee2af-af91-46bd-ab6b-0289fa3c0ef3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feca6278be904ef8b0dc580a67e477f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:23:16.990698 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50ee2af-af91-46bd-ab6b-0289fa3c0ef3 service cinder] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:23:16.990982 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50ee2af-af91-46bd-ab6b-0289fa3c0ef3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:23:16.991256 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50ee2af-af91-46bd-ab6b-0289fa3c0ef3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:23:17.000068 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50ee2af-af91-46bd-ab6b-0289fa3c0ef3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:23:17.001484 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50ee2af-af91-46bd-ab6b-0289fa3c0ef3 service cinder] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:23:17.011107 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1187/2350] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:23:16 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 23 15:23:17.012167 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5382e77b-d7cc-45ea-a09d-334d40cb12e3 None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:23:17.012504 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5382e77b-d7cc-45ea-a09d-334d40cb12e3 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:23:17.013211 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5382e77b-d7cc-45ea-a09d-334d40cb12e3 None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:23:17.014036 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1188/2351] 10.222.0.55 () {54 vars in 921 bytes} [Fri Sep 23 15:23:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:23:17.018990 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d404d8f-50f1-4b51-8c0d-5f7e77b05fcd None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:23:17.019281 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d404d8f-50f1-4b51-8c0d-5f7e77b05fcd None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:23:17.019562 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d404d8f-50f1-4b51-8c0d-5f7e77b05fcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:23:17.036592 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6828ce-c737-42bb-afeb-e2b7d803835e None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:23:17.041295 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-1d404d8f-50f1-4b51-8c0d-5f7e77b05fcd None None] MFA Rules not processed for user `d259178570864ffeb936ee10dfec6b28`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:23:17.042524 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-1d404d8f-50f1-4b51-8c0d-5f7e77b05fcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:23:17.044684 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296d73e2-d5ea-4855-a35e-cf74d1896c65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:23:17.045790 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296d73e2-d5ea-4855-a35e-cf74d1896c65 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:23:17.049015 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296d73e2-d5ea-4855-a35e-cf74d1896c65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:23:17.049015 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296d73e2-d5ea-4855-a35e-cf74d1896c65 admin admin] PATH_INFO: `/v3/users/668fbf0ee9f444ac84b075a2ec397af2` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:23:17.049015 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296d73e2-d5ea-4855-a35e-cf74d1896c65 admin admin] RBAC: Authorizing `identity:delete_user(user_id=668fbf0ee9f444ac84b075a2ec397af2)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:23:17.049798 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296d73e2-d5ea-4855-a35e-cf74d1896c65 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:23:17.092310 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:23:17.103162 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1189/2352] 10.222.0.55 () {58 vars in 1018 bytes} [Fri Sep 23 15:23:17 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 15:23:17.103815 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d404d8f-50f1-4b51-8c0d-5f7e77b05fcd None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:23:17.115765 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d404d8f-50f1-4b51-8c0d-5f7e77b05fcd None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:23:17.120368 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4830ae-883e-4827-85db-798d0b6b1ba3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:23:17.120998 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4830ae-883e-4827-85db-798d0b6b1ba3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:23:17.121282 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4830ae-883e-4827-85db-798d0b6b1ba3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:23:17.121602 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4830ae-883e-4827-85db-798d0b6b1ba3 admin admin] PATH_INFO: `/v3/users/260ef126a1e74fbe8bd23bc977bf97ab` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:23:17.122963 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4830ae-883e-4827-85db-798d0b6b1ba3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=260ef126a1e74fbe8bd23bc977bf97ab)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:23:17.124919 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4830ae-883e-4827-85db-798d0b6b1ba3 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:23:17.349055 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-296d73e2-d5ea-4855-a35e-cf74d1896c65 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '668fbf0ee9f444ac84b075a2ec397af2'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:23:17.361328 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-296d73e2-d5ea-4855-a35e-cf74d1896c65 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '668fbf0ee9f444ac84b075a2ec397af2'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:23:17.368429 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-296d73e2-d5ea-4855-a35e-cf74d1896c65 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '668fbf0ee9f444ac84b075a2ec397af2'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:23:17.431394 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-9d4830ae-883e-4827-85db-798d0b6b1ba3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '260ef126a1e74fbe8bd23bc977bf97ab'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:23:17.453256 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-9d4830ae-883e-4827-85db-798d0b6b1ba3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '260ef126a1e74fbe8bd23bc977bf97ab'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:23:17.460750 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-9d4830ae-883e-4827-85db-798d0b6b1ba3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '260ef126a1e74fbe8bd23bc977bf97ab'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:23:17.484675 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-296d73e2-d5ea-4855-a35e-cf74d1896c65 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '668fbf0ee9f444ac84b075a2ec397af2'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:23:17.485669 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-296d73e2-d5ea-4855-a35e-cf74d1896c65 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '668fbf0ee9f444ac84b075a2ec397af2'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:23:17.492556 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1164/2353] 10.222.0.55 () {58 vars in 1224 bytes} [Fri Sep 23 15:23:16 2022] DELETE /identity/v3/users/668fbf0ee9f444ac84b075a2ec397af2 => generated 0 bytes in 534 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:23:17.493594 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296d73e2-d5ea-4855-a35e-cf74d1896c65 admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:23:17.494920 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-296d73e2-d5ea-4855-a35e-cf74d1896c65 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:23:17.557100 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296d73e2-d5ea-4855-a35e-cf74d1896c65 admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:23:17.560771 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b9e762-9485-4235-9539-52b36bb7a454 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd259178570864ffeb936ee10dfec6b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:23:17.561380 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b9e762-9485-4235-9539-52b36bb7a454 service neutron] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:23:17.561682 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b9e762-9485-4235-9539-52b36bb7a454 service neutron] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:23:17.561931 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b9e762-9485-4235-9539-52b36bb7a454 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:23:17.563827 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-c1b9e762-9485-4235-9539-52b36bb7a454 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:23:17.573369 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b9e762-9485-4235-9539-52b36bb7a454 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:23:17.575357 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b9e762-9485-4235-9539-52b36bb7a454 service neutron] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:23:17.581731 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-9d4830ae-883e-4827-85db-798d0b6b1ba3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '260ef126a1e74fbe8bd23bc977bf97ab'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:23:17.582448 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-9d4830ae-883e-4827-85db-798d0b6b1ba3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '260ef126a1e74fbe8bd23bc977bf97ab'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:23:17.583348 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1165/2354] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:23:17 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:23:17.584574 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157dcc2f-0b77-4cda-8d8a-5a166e9c1bd8 None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:23:17.584928 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157dcc2f-0b77-4cda-8d8a-5a166e9c1bd8 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:23:17.585213 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157dcc2f-0b77-4cda-8d8a-5a166e9c1bd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:23:17.590156 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1190/2355] 10.222.0.55 () {58 vars in 1224 bytes} [Fri Sep 23 15:23:17 2022] DELETE /identity/v3/users/260ef126a1e74fbe8bd23bc977bf97ab => generated 0 bytes in 487 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:23:17.596018 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbad4470-5817-46da-a779-e66dd479bbe5 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:23:17.598955 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbad4470-5817-46da-a779-e66dd479bbe5 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:23:17.598955 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbad4470-5817-46da-a779-e66dd479bbe5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:23:17.612636 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-157dcc2f-0b77-4cda-8d8a-5a166e9c1bd8 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:23:17.613942 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-157dcc2f-0b77-4cda-8d8a-5a166e9c1bd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:23:17.618084 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-fbad4470-5817-46da-a779-e66dd479bbe5 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:23:17.619766 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-fbad4470-5817-46da-a779-e66dd479bbe5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:23:17.692280 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1166/2356] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:23:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:23:17.692764 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1191/2357] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:23:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:23:17.693561 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb732fc0-2a46-4bf2-a500-a33ac440153e None None] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:23:17.693848 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb732fc0-2a46-4bf2-a500-a33ac440153e None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:23:17.694121 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb732fc0-2a46-4bf2-a500-a33ac440153e None None] PATH_INFO: `/` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:23:17.695375 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1167/2358] 10.222.0.55 () {54 vars in 921 bytes} [Fri Sep 23 15:23:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:23:17.703051 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbad4470-5817-46da-a779-e66dd479bbe5 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:23:17.705677 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-fbad4470-5817-46da-a779-e66dd479bbe5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:23:17.705784 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb732fc0-2a46-4bf2-a500-a33ac440153e None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:23:17.706957 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-eb732fc0-2a46-4bf2-a500-a33ac440153e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:23:17.767171 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbad4470-5817-46da-a779-e66dd479bbe5 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:23:17.770963 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6206e90-b0cd-42e0-ae04-bbca08366446 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd259178570864ffeb936ee10dfec6b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:23:17.771102 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb732fc0-2a46-4bf2-a500-a33ac440153e None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:23:17.771580 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6206e90-b0cd-42e0-ae04-bbca08366446 service neutron] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:23:17.771862 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6206e90-b0cd-42e0-ae04-bbca08366446 service neutron] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:23:17.772167 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6206e90-b0cd-42e0-ae04-bbca08366446 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:23:17.774722 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e147954e-b155-4201-9c8f-26aef1f43dad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd259178570864ffeb936ee10dfec6b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:23:17.775360 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e147954e-b155-4201-9c8f-26aef1f43dad service neutron] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:23:17.775643 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e147954e-b155-4201-9c8f-26aef1f43dad service neutron] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:23:17.775918 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e147954e-b155-4201-9c8f-26aef1f43dad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:23:17.781981 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6206e90-b0cd-42e0-ae04-bbca08366446 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:23:17.783495 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6206e90-b0cd-42e0-ae04-bbca08366446 service neutron] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:23:17.786618 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e147954e-b155-4201-9c8f-26aef1f43dad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:23:17.788112 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e147954e-b155-4201-9c8f-26aef1f43dad service neutron] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:23:17.793283 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1192/2359] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:23:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 23 15:23:17.797405 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb87f1eb-56f3-4cb9-ad97-f74d00bf6282 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:23:17.797749 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb87f1eb-56f3-4cb9-ad97-f74d00bf6282 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:23:17.798054 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb87f1eb-56f3-4cb9-ad97-f74d00bf6282 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:23:17.799391 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1168/2360] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:23:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:23:17.829547 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-fb87f1eb-56f3-4cb9-ad97-f74d00bf6282 None None] MFA Rules not processed for user `b3c6c7a1325a47be924268e1aa1a7a18`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:23:17.830824 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-fb87f1eb-56f3-4cb9-ad97-f74d00bf6282 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:23:17.919295 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e147954e-b155-4201-9c8f-26aef1f43dad service neutron] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:23:17.921900 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-e147954e-b155-4201-9c8f-26aef1f43dad service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:23:17.936756 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e147954e-b155-4201-9c8f-26aef1f43dad service neutron] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:23:17.944021 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c58b337-1721-404b-a3bc-c032a0b1c139 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:23:17.944021 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c58b337-1721-404b-a3bc-c032a0b1c139 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:23:17.944021 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c58b337-1721-404b-a3bc-c032a0b1c139 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:23:17.944021 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c58b337-1721-404b-a3bc-c032a0b1c139 admin admin] PATH_INFO: `/v3/users/42b46c6aac51477da1ead5348445e2ea` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:23:17.945039 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c58b337-1721-404b-a3bc-c032a0b1c139 admin admin] RBAC: Authorizing `identity:delete_user(user_id=42b46c6aac51477da1ead5348445e2ea)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:23:17.947002 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c58b337-1721-404b-a3bc-c032a0b1c139 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:23:17.947510 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1193/2361] 10.222.0.55 () {58 vars in 1018 bytes} [Fri Sep 23 15:23:17 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:23:17.959036 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb87f1eb-56f3-4cb9-ad97-f74d00bf6282 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:23:17.970813 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb87f1eb-56f3-4cb9-ad97-f74d00bf6282 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:23:17.974758 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beaab38c-0086-4772-af4b-df80c0cc5db9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd259178570864ffeb936ee10dfec6b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:23:17.975450 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaab38c-0086-4772-af4b-df80c0cc5db9 service neutron] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:23:17.975828 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaab38c-0086-4772-af4b-df80c0cc5db9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:23:17.976157 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaab38c-0086-4772-af4b-df80c0cc5db9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:23:17.988741 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beaab38c-0086-4772-af4b-df80c0cc5db9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:23:17.990196 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beaab38c-0086-4772-af4b-df80c0cc5db9 service neutron] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:23:18.006181 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1194/2362] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:23:17 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:23:18.305039 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-4c58b337-1721-404b-a3bc-c032a0b1c139 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '42b46c6aac51477da1ead5348445e2ea'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:23:18.334625 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-4c58b337-1721-404b-a3bc-c032a0b1c139 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '42b46c6aac51477da1ead5348445e2ea'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:23:18.344084 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-4c58b337-1721-404b-a3bc-c032a0b1c139 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '42b46c6aac51477da1ead5348445e2ea'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:23:18.515919 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-4c58b337-1721-404b-a3bc-c032a0b1c139 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '42b46c6aac51477da1ead5348445e2ea'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:23:18.516585 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-4c58b337-1721-404b-a3bc-c032a0b1c139 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '42b46c6aac51477da1ead5348445e2ea'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:23:18.524140 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beaab38c-0086-4772-af4b-df80c0cc5db9 service neutron] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:23:18.524559 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1169/2363] 10.222.0.55 () {58 vars in 1224 bytes} [Fri Sep 23 15:23:17 2022] DELETE /identity/v3/users/42b46c6aac51477da1ead5348445e2ea => generated 0 bytes in 606 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:23:18.525926 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-beaab38c-0086-4772-af4b-df80c0cc5db9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:23:18.534042 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c58b337-1721-404b-a3bc-c032a0b1c139 admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:23:18.539395 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-4c58b337-1721-404b-a3bc-c032a0b1c139 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:23:18.549348 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beaab38c-0086-4772-af4b-df80c0cc5db9 service neutron] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:23:18.616421 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:INSERT=2,keystone:SELECT=83,keystone:DELETE=15 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:23:18.665072 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c58b337-1721-404b-a3bc-c032a0b1c139 admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:23:18.670809 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7584b6a-d4c1-4bf1-9fab-95ec5b1c16a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:23:18.671533 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7584b6a-d4c1-4bf1-9fab-95ec5b1c16a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:23:18.671818 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7584b6a-d4c1-4bf1-9fab-95ec5b1c16a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:23:18.672128 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7584b6a-d4c1-4bf1-9fab-95ec5b1c16a3 admin admin] PATH_INFO: `/v3/projects/4073032ebdc54b3b95e818cf3c5a3878` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:23:18.673325 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7584b6a-d4c1-4bf1-9fab-95ec5b1c16a3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4073032ebdc54b3b95e818cf3c5a3878)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:23:18.673795 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922efad4-3ae2-4b6d-a823-fbed644adb6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:23:18.674455 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922efad4-3ae2-4b6d-a823-fbed644adb6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:23:18.674888 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922efad4-3ae2-4b6d-a823-fbed644adb6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:23:18.675176 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922efad4-3ae2-4b6d-a823-fbed644adb6e admin admin] PATH_INFO: `/v3/projects/86aa75dc0b724749ab4cd37b65666154` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:23:18.676131 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7584b6a-d4c1-4bf1-9fab-95ec5b1c16a3 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:23:18.676611 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922efad4-3ae2-4b6d-a823-fbed644adb6e admin admin] RBAC: Authorizing `identity:delete_project(project_id=86aa75dc0b724749ab4cd37b65666154)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:23:18.678742 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922efad4-3ae2-4b6d-a823-fbed644adb6e admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:23:18.852223 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-f7584b6a-d4c1-4bf1-9fab-95ec5b1c16a3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4073032ebdc54b3b95e818cf3c5a3878', 'request_id': 'req-f7584b6a-d4c1-4bf1-9fab-95ec5b1c16a3'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:23:18.863422 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-922efad4-3ae2-4b6d-a823-fbed644adb6e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '86aa75dc0b724749ab4cd37b65666154', 'request_id': 'req-922efad4-3ae2-4b6d-a823-fbed644adb6e'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:23:18.863686 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-f7584b6a-d4c1-4bf1-9fab-95ec5b1c16a3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4073032ebdc54b3b95e818cf3c5a3878', 'request_id': 'req-f7584b6a-d4c1-4bf1-9fab-95ec5b1c16a3'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:23:18.871490 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-922efad4-3ae2-4b6d-a823-fbed644adb6e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '86aa75dc0b724749ab4cd37b65666154', 'request_id': 'req-922efad4-3ae2-4b6d-a823-fbed644adb6e'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:23:18.878622 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-922efad4-3ae2-4b6d-a823-fbed644adb6e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '86aa75dc0b724749ab4cd37b65666154', 'request_id': 'req-922efad4-3ae2-4b6d-a823-fbed644adb6e'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:23:18.970607 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-f7584b6a-d4c1-4bf1-9fab-95ec5b1c16a3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4073032ebdc54b3b95e818cf3c5a3878', 'request_id': 'req-f7584b6a-d4c1-4bf1-9fab-95ec5b1c16a3'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:23:18.986463 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-f7584b6a-d4c1-4bf1-9fab-95ec5b1c16a3 admin admin] The token cache is being invalidate because project 4073032ebdc54b3b95e818cf3c5a3878 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81993) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 15:23:18.986823 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-f7584b6a-d4c1-4bf1-9fab-95ec5b1c16a3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:23:18.987932 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-f7584b6a-d4c1-4bf1-9fab-95ec5b1c16a3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:23:18.989772 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1170/2364] 10.222.0.55 () {58 vars in 1233 bytes} [Fri Sep 23 15:23:18 2022] DELETE /identity/v3/projects/4073032ebdc54b3b95e818cf3c5a3878 => generated 0 bytes in 458 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 15:23:18.991310 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a6a4cb-18a3-4cec-8212-1e40f8410de0 None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:23:18.991694 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a6a4cb-18a3-4cec-8212-1e40f8410de0 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:23:18.992037 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a6a4cb-18a3-4cec-8212-1e40f8410de0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:23:19.015816 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-27a6a4cb-18a3-4cec-8212-1e40f8410de0 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:23:19.017498 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-27a6a4cb-18a3-4cec-8212-1e40f8410de0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:23:19.102338 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1171/2365] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:23:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:23:19.114366 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-922efad4-3ae2-4b6d-a823-fbed644adb6e admin admin] The token cache is being invalidate because project 86aa75dc0b724749ab4cd37b65666154 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81994) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 15:23:19.118797 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-922efad4-3ae2-4b6d-a823-fbed644adb6e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:23:19.118797 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-922efad4-3ae2-4b6d-a823-fbed644adb6e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:23:19.119663 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a6a4cb-18a3-4cec-8212-1e40f8410de0 None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:23:19.121331 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1195/2366] 10.222.0.55 () {58 vars in 1233 bytes} [Fri Sep 23 15:23:18 2022] DELETE /identity/v3/projects/86aa75dc0b724749ab4cd37b65666154 => generated 0 bytes in 598 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:23:19.122650 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-27a6a4cb-18a3-4cec-8212-1e40f8410de0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:23:19.126645 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17c2029-5c6c-403e-927f-791378625bc3 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:23:19.126948 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17c2029-5c6c-403e-927f-791378625bc3 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:23:19.127883 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17c2029-5c6c-403e-927f-791378625bc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:23:19.167532 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-b17c2029-5c6c-403e-927f-791378625bc3 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:23:19.169360 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-b17c2029-5c6c-403e-927f-791378625bc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:23:19.183070 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1196/2367] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:23:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:23:19.194378 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4707685-e968-41de-9cb3-fd13ed4f488d None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:23:19.194812 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4707685-e968-41de-9cb3-fd13ed4f488d None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:23:19.194967 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4707685-e968-41de-9cb3-fd13ed4f488d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:23:19.231953 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-e4707685-e968-41de-9cb3-fd13ed4f488d None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:23:19.233584 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-e4707685-e968-41de-9cb3-fd13ed4f488d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:23:19.237391 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a6a4cb-18a3-4cec-8212-1e40f8410de0 None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:23:19.241401 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed2f870-1b37-44ec-b07a-f88791fcd05f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd259178570864ffeb936ee10dfec6b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:23:19.241735 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1197/2368] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:23:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:23:19.242501 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed2f870-1b37-44ec-b07a-f88791fcd05f service neutron] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:23:19.242922 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed2f870-1b37-44ec-b07a-f88791fcd05f service neutron] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:23:19.243271 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed2f870-1b37-44ec-b07a-f88791fcd05f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:23:19.246195 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-7ed2f870-1b37-44ec-b07a-f88791fcd05f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:23:19.249158 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4707685-e968-41de-9cb3-fd13ed4f488d None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:23:19.259209 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4707685-e968-41de-9cb3-fd13ed4f488d None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:23:19.269522 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed2f870-1b37-44ec-b07a-f88791fcd05f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:23:19.269522 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed2f870-1b37-44ec-b07a-f88791fcd05f service neutron] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:23:19.269736 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab8d46a-c32d-40e0-b102-a6fb3c922378 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:23:19.269736 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab8d46a-c32d-40e0-b102-a6fb3c922378 admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:23:19.269736 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab8d46a-c32d-40e0-b102-a6fb3c922378 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:23:19.269736 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab8d46a-c32d-40e0-b102-a6fb3c922378 admin admin] PATH_INFO: `/v3/domains` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:23:19.270126 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab8d46a-c32d-40e0-b102-a6fb3c922378 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:23:19.270126 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab8d46a-c32d-40e0-b102-a6fb3c922378 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:23:19.272036 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab8d46a-c32d-40e0-b102-a6fb3c922378 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:23:19.276104 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1172/2369] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:23:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:23:19.289690 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1198/2370] 10.222.0.55 () {58 vars in 1153 bytes} [Fri Sep 23 15:23:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:23:19.294273 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed2f870-1b37-44ec-b07a-f88791fcd05f service neutron] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:23:19.305898 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed2f870-1b37-44ec-b07a-f88791fcd05f service neutron] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:23:19.311361 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e1ca3c-dbc5-463e-ba86-e5a778a9b40e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:23:19.312244 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e1ca3c-dbc5-463e-ba86-e5a778a9b40e admin admin] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:23:19.312244 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e1ca3c-dbc5-463e-ba86-e5a778a9b40e admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:23:19.312520 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e1ca3c-dbc5-463e-ba86-e5a778a9b40e admin admin] PATH_INFO: `/v3/projects` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:23:19.313113 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e1ca3c-dbc5-463e-ba86-e5a778a9b40e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:23:19.315067 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e1ca3c-dbc5-463e-ba86-e5a778a9b40e admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:23:19.435721 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1173/2371] 10.222.0.55 () {60 vars in 1153 bytes} [Fri Sep 23 15:23:19 2022] POST /identity/v3/projects => generated 383 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 23 15:23:19.440362 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab8d46a-c32d-40e0-b102-a6fb3c922378 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:23:19.452079 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab8d46a-c32d-40e0-b102-a6fb3c922378 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:23:19.511447 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83dd26a-4e3c-4770-a3bf-d7b39f9818e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:23:19.512956 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83dd26a-4e3c-4770-a3bf-d7b39f9818e8 admin admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:23:19.512956 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83dd26a-4e3c-4770-a3bf-d7b39f9818e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:23:19.513222 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83dd26a-4e3c-4770-a3bf-d7b39f9818e8 admin admin] PATH_INFO: `/v3/users` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:23:19.513979 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83dd26a-4e3c-4770-a3bf-d7b39f9818e8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:23:19.515923 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83dd26a-4e3c-4770-a3bf-d7b39f9818e8 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:23:19.587709 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:23:19.645466 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1199/2372] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Sep 23 15:23:19 2022] POST /identity/v3/users => generated 292 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 23 15:23:19.646659 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e1ca3c-dbc5-463e-ba86-e5a778a9b40e admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:23:19.658069 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e1ca3c-dbc5-463e-ba86-e5a778a9b40e admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:23:19.662954 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee60ee47-56a8-4e19-8b2b-3992ceda6a9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:23:19.663592 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee60ee47-56a8-4e19-8b2b-3992ceda6a9e admin admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:23:19.663871 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee60ee47-56a8-4e19-8b2b-3992ceda6a9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:23:19.664214 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee60ee47-56a8-4e19-8b2b-3992ceda6a9e admin admin] PATH_INFO: `/v3/roles` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:23:19.664885 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee60ee47-56a8-4e19-8b2b-3992ceda6a9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:23:19.666510 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee60ee47-56a8-4e19-8b2b-3992ceda6a9e admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:23:19.677800 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1174/2373] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Sep 23 15:23:19 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:23:19.682080 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83dd26a-4e3c-4770-a3bf-d7b39f9818e8 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:23:19.694627 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83dd26a-4e3c-4770-a3bf-d7b39f9818e8 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:23:19.699411 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6f8cda-70c6-4b78-a897-524b198833a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:23:19.700679 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6f8cda-70c6-4b78-a897-524b198833a7 admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:23:19.700996 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6f8cda-70c6-4b78-a897-524b198833a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:23:19.701271 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6f8cda-70c6-4b78-a897-524b198833a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:23:19.701914 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6f8cda-70c6-4b78-a897-524b198833a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:23:19.703551 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6f8cda-70c6-4b78-a897-524b198833a7 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:23:19.714918 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1200/2374] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Sep 23 15:23:19 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:23:19.719347 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee60ee47-56a8-4e19-8b2b-3992ceda6a9e admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:23:19.729032 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee60ee47-56a8-4e19-8b2b-3992ceda6a9e admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:23:19.738946 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ca5874-2022-44d8-afb5-7fd3e3c39c1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:23:19.739374 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ca5874-2022-44d8-afb5-7fd3e3c39c1c admin admin] REQUEST_METHOD: `PUT` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:23:19.739657 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ca5874-2022-44d8-afb5-7fd3e3c39c1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:23:19.739970 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ca5874-2022-44d8-afb5-7fd3e3c39c1c admin admin] PATH_INFO: `/v3/projects/efbab8ef4fbd4c61811d0fc046290937/users/deda513f4c6448368cfa79203cf0d8d8/roles/af257be2dee54d6b942f58650196582c` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:23:19.757635 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ca5874-2022-44d8-afb5-7fd3e3c39c1c admin admin] RBAC: Authorizing `identity:create_grant(project_id=efbab8ef4fbd4c61811d0fc046290937, user_id=deda513f4c6448368cfa79203cf0d8d8, role_id=af257be2dee54d6b942f58650196582c)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:23:19.760374 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ca5874-2022-44d8-afb5-7fd3e3c39c1c admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:23:19.796538 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6f8cda-70c6-4b78-a897-524b198833a7 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:23:19.798616 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-ef6f8cda-70c6-4b78-a897-524b198833a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:23:19.814031 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6f8cda-70c6-4b78-a897-524b198833a7 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:23:19.819256 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c5da30-adde-4693-a944-9ecd7b3cb0be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:23:19.819915 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c5da30-adde-4693-a944-9ecd7b3cb0be admin admin] REQUEST_METHOD: `DELETE` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:23:19.820347 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c5da30-adde-4693-a944-9ecd7b3cb0be admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:23:19.820626 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c5da30-adde-4693-a944-9ecd7b3cb0be admin admin] PATH_INFO: `/v3/projects/10fdbae91862410ebdcdff525598875e` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:23:19.821769 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c5da30-adde-4693-a944-9ecd7b3cb0be admin admin] RBAC: Authorizing `identity:delete_project(project_id=10fdbae91862410ebdcdff525598875e)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:23:19.823717 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c5da30-adde-4693-a944-9ecd7b3cb0be admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:23:19.920079 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1175/2375] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Sep 23 15:23:19 2022] PUT /identity/v3/projects/efbab8ef4fbd4c61811d0fc046290937/users/deda513f4c6448368cfa79203cf0d8d8/roles/af257be2dee54d6b942f58650196582c => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:23:19.936386 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c83203-08f7-4ebb-b325-6622502458eb None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:23:19.936676 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c83203-08f7-4ebb-b325-6622502458eb None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:23:19.936960 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c83203-08f7-4ebb-b325-6622502458eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:23:19.957486 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-e2c83203-08f7-4ebb-b325-6622502458eb None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:23:19.958751 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-e2c83203-08f7-4ebb-b325-6622502458eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:23:19.973398 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-72c5da30-adde-4693-a944-9ecd7b3cb0be admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '10fdbae91862410ebdcdff525598875e', 'request_id': 'req-72c5da30-adde-4693-a944-9ecd7b3cb0be'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:23:19.979944 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-72c5da30-adde-4693-a944-9ecd7b3cb0be admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '10fdbae91862410ebdcdff525598875e', 'request_id': 'req-72c5da30-adde-4693-a944-9ecd7b3cb0be'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:23:19.986814 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-72c5da30-adde-4693-a944-9ecd7b3cb0be admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '10fdbae91862410ebdcdff525598875e', 'request_id': 'req-72c5da30-adde-4693-a944-9ecd7b3cb0be'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:23:20.021779 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1176/2376] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:23:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 15:23:20.032873 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c83203-08f7-4ebb-b325-6622502458eb None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:23:20.034249 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-e2c83203-08f7-4ebb-b325-6622502458eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:23:20.097506 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c83203-08f7-4ebb-b325-6622502458eb None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:23:20.101308 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145e6355-ab47-4db1-82b6-e06c44118cb3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd259178570864ffeb936ee10dfec6b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:23:20.101942 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145e6355-ab47-4db1-82b6-e06c44118cb3 service neutron] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:23:20.105732 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145e6355-ab47-4db1-82b6-e06c44118cb3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:23:20.105732 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145e6355-ab47-4db1-82b6-e06c44118cb3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:23:20.113355 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145e6355-ab47-4db1-82b6-e06c44118cb3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:23:20.114791 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145e6355-ab47-4db1-82b6-e06c44118cb3 service neutron] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:23:20.118450 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-72c5da30-adde-4693-a944-9ecd7b3cb0be admin admin] The token cache is being invalidate because project 10fdbae91862410ebdcdff525598875e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81994) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 15:23:20.118903 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-72c5da30-adde-4693-a944-9ecd7b3cb0be admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:23:20.119436 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-72c5da30-adde-4693-a944-9ecd7b3cb0be admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:23:20.121564 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1201/2377] 10.222.0.55 () {58 vars in 1233 bytes} [Fri Sep 23 15:23:19 2022] DELETE /identity/v3/projects/10fdbae91862410ebdcdff525598875e => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:23:20.125882 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c5da30-adde-4693-a944-9ecd7b3cb0be admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:23:20.127880 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1177/2378] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:23:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 23 15:23:20.132041 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-72c5da30-adde-4693-a944-9ecd7b3cb0be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:23:20.145563 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c5da30-adde-4693-a944-9ecd7b3cb0be admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:23:20.150242 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0dfa94e-b645-4d21-98e2-33a4b7473151 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:23:20.150879 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dfa94e-b645-4d21-98e2-33a4b7473151 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:23:20.151693 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dfa94e-b645-4d21-98e2-33a4b7473151 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:23:20.151693 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dfa94e-b645-4d21-98e2-33a4b7473151 admin admin] PATH_INFO: `/v3/users/07c58b7fda4146ef969b0c0d953ea95e` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:23:20.153263 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0dfa94e-b645-4d21-98e2-33a4b7473151 admin admin] RBAC: Authorizing `identity:delete_user(user_id=07c58b7fda4146ef969b0c0d953ea95e)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:23:20.155307 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0dfa94e-b645-4d21-98e2-33a4b7473151 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:23:20.473248 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-b0dfa94e-b645-4d21-98e2-33a4b7473151 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '07c58b7fda4146ef969b0c0d953ea95e'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:23:20.490009 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-b0dfa94e-b645-4d21-98e2-33a4b7473151 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '07c58b7fda4146ef969b0c0d953ea95e'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:23:20.504598 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-b0dfa94e-b645-4d21-98e2-33a4b7473151 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '07c58b7fda4146ef969b0c0d953ea95e'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:23:20.610246 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-b0dfa94e-b645-4d21-98e2-33a4b7473151 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '07c58b7fda4146ef969b0c0d953ea95e'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:23:20.611094 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-b0dfa94e-b645-4d21-98e2-33a4b7473151 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '07c58b7fda4146ef969b0c0d953ea95e'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:23:20.618369 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1202/2379] 10.222.0.55 () {58 vars in 1224 bytes} [Fri Sep 23 15:23:20 2022] DELETE /identity/v3/users/07c58b7fda4146ef969b0c0d953ea95e => generated 0 bytes in 493 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 15:23:21.162657 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145e6355-ab47-4db1-82b6-e06c44118cb3 service neutron] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:23:21.172802 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145e6355-ab47-4db1-82b6-e06c44118cb3 service neutron] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:23:21.244323 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcab5e5-e0f4-485c-85a6-053b981943fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:23:21.244966 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcab5e5-e0f4-485c-85a6-053b981943fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:23:21.247118 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcab5e5-e0f4-485c-85a6-053b981943fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:23:21.247118 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcab5e5-e0f4-485c-85a6-053b981943fb admin admin] PATH_INFO: `/v3/projects/6450c1191bf148279d65ed2c73baba58` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:23:21.247118 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcab5e5-e0f4-485c-85a6-053b981943fb admin admin] RBAC: Authorizing `identity:delete_project(project_id=6450c1191bf148279d65ed2c73baba58)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:23:21.248700 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcab5e5-e0f4-485c-85a6-053b981943fb admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:23:21.401009 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-2dcab5e5-e0f4-485c-85a6-053b981943fb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6450c1191bf148279d65ed2c73baba58', 'request_id': 'req-2dcab5e5-e0f4-485c-85a6-053b981943fb'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:23:21.407428 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-2dcab5e5-e0f4-485c-85a6-053b981943fb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6450c1191bf148279d65ed2c73baba58', 'request_id': 'req-2dcab5e5-e0f4-485c-85a6-053b981943fb'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:23:21.490233 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-2dcab5e5-e0f4-485c-85a6-053b981943fb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6450c1191bf148279d65ed2c73baba58', 'request_id': 'req-2dcab5e5-e0f4-485c-85a6-053b981943fb'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:23:21.503468 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-2dcab5e5-e0f4-485c-85a6-053b981943fb admin admin] The token cache is being invalidate because project 6450c1191bf148279d65ed2c73baba58 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81993) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 15:23:21.503694 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-2dcab5e5-e0f4-485c-85a6-053b981943fb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:23:21.504772 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-2dcab5e5-e0f4-485c-85a6-053b981943fb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:23:21.506646 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1178/2380] 10.222.0.55 () {58 vars in 1233 bytes} [Fri Sep 23 15:23:21 2022] DELETE /identity/v3/projects/6450c1191bf148279d65ed2c73baba58 => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:23:22.433610 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=95,keystone:INSERT=1,keystone:DELETE=4 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:23:22.507960 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:DELETE=2,keystone:SELECT=93,keystone:INSERT=5 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:23:24.136618 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:23:24.166214 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bbf1a0-511d-4511-8792-a0f1fddbb1f7 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:23:24.166504 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bbf1a0-511d-4511-8792-a0f1fddbb1f7 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:23:24.166783 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bbf1a0-511d-4511-8792-a0f1fddbb1f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:23:24.196254 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-e5bbf1a0-511d-4511-8792-a0f1fddbb1f7 None None] MFA Rules not processed for user `68206a6b4f244d45bf7bf1504396e430`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:23:24.200785 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:23:24.203263 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-e5bbf1a0-511d-4511-8792-a0f1fddbb1f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:23:24.300730 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1203/2381] 10.222.0.55 () {58 vars in 1009 bytes} [Fri Sep 23 15:23:24 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:23:24.325397 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcab5e5-e0f4-485c-85a6-053b981943fb admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:23:24.325397 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-2dcab5e5-e0f4-485c-85a6-053b981943fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:23:24.437768 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcab5e5-e0f4-485c-85a6-053b981943fb admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:23:24.445039 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86697ff-8e64-4c58-9ea7-d6d8095bc5a6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2228c0b49a0249bbaf524a8c4e9e108b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:23:24.446152 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86697ff-8e64-4c58-9ea7-d6d8095bc5a6 service swift] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:23:24.446771 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86697ff-8e64-4c58-9ea7-d6d8095bc5a6 service swift] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:23:24.447549 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86697ff-8e64-4c58-9ea7-d6d8095bc5a6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:23:24.460762 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86697ff-8e64-4c58-9ea7-d6d8095bc5a6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:23:24.462406 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86697ff-8e64-4c58-9ea7-d6d8095bc5a6 service swift] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:23:24.472821 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1179/2382] 10.222.0.55 () {60 vars in 1384 bytes} [Fri Sep 23 15:23:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 23 15:23:25.199443 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:23:25.948122 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:23:27.415994 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:23:27.514592 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:23:29.368117 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad94d9a3-429d-403d-9edb-dce1411e7f6e None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:23:29.368117 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad94d9a3-429d-403d-9edb-dce1411e7f6e None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:23:29.368117 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad94d9a3-429d-403d-9edb-dce1411e7f6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:23:29.400609 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-ad94d9a3-429d-403d-9edb-dce1411e7f6e None None] MFA Rules not processed for user `deda513f4c6448368cfa79203cf0d8d8`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:23:29.407923 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-ad94d9a3-429d-403d-9edb-dce1411e7f6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:23:29.483197 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86697ff-8e64-4c58-9ea7-d6d8095bc5a6 service swift] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:23:29.484405 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1204/2383] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:23:29 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:23:29.485672 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-c86697ff-8e64-4c58-9ea7-d6d8095bc5a6 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:23:29.610497 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86697ff-8e64-4c58-9ea7-d6d8095bc5a6 service swift] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:23:29.615384 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d947653b-e98a-4bf4-aad1-73a6e8b9811f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:23:29.618556 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d947653b-e98a-4bf4-aad1-73a6e8b9811f admin admin] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:23:29.618556 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d947653b-e98a-4bf4-aad1-73a6e8b9811f admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:23:29.618556 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d947653b-e98a-4bf4-aad1-73a6e8b9811f admin admin] PATH_INFO: `/v3/projects` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:23:29.618556 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d947653b-e98a-4bf4-aad1-73a6e8b9811f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:23:29.619309 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d947653b-e98a-4bf4-aad1-73a6e8b9811f admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:23:29.686024 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:23:29.728092 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1180/2384] 10.222.0.55 () {60 vars in 1153 bytes} [Fri Sep 23 15:23:29 2022] POST /identity/v3/projects => generated 383 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:23:29.733053 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad94d9a3-429d-403d-9edb-dce1411e7f6e None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:23:29.750269 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad94d9a3-429d-403d-9edb-dce1411e7f6e None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:23:29.860362 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89020965-f8a8-4563-a0ca-5b0f8f22929c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:23:29.861061 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89020965-f8a8-4563-a0ca-5b0f8f22929c admin admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:23:29.861456 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89020965-f8a8-4563-a0ca-5b0f8f22929c admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:23:29.861710 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89020965-f8a8-4563-a0ca-5b0f8f22929c admin admin] PATH_INFO: `/v3/users` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:23:29.862370 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89020965-f8a8-4563-a0ca-5b0f8f22929c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:23:29.864665 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89020965-f8a8-4563-a0ca-5b0f8f22929c admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:23:30.020987 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1205/2385] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Sep 23 15:23:29 2022] POST /identity/v3/users => generated 298 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 23 15:23:30.022442 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d947653b-e98a-4bf4-aad1-73a6e8b9811f admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:23:30.033990 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d947653b-e98a-4bf4-aad1-73a6e8b9811f admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:23:30.048592 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54287ea0-ed4c-4c02-9007-8a0e8e3b9200 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:23:30.056712 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54287ea0-ed4c-4c02-9007-8a0e8e3b9200 admin admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:23:30.056712 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54287ea0-ed4c-4c02-9007-8a0e8e3b9200 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:23:30.056712 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54287ea0-ed4c-4c02-9007-8a0e8e3b9200 admin admin] PATH_INFO: `/v3/roles` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:23:30.056712 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54287ea0-ed4c-4c02-9007-8a0e8e3b9200 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:23:30.056712 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54287ea0-ed4c-4c02-9007-8a0e8e3b9200 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:23:30.092401 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1181/2386] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Sep 23 15:23:30 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:23:30.096664 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89020965-f8a8-4563-a0ca-5b0f8f22929c admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:23:30.118432 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89020965-f8a8-4563-a0ca-5b0f8f22929c admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:23:30.137474 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99565bcd-5a92-4163-9d0d-177a188499ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:23:30.138612 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99565bcd-5a92-4163-9d0d-177a188499ff admin admin] REQUEST_METHOD: `PUT` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:23:30.138971 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99565bcd-5a92-4163-9d0d-177a188499ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:23:30.139339 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99565bcd-5a92-4163-9d0d-177a188499ff admin admin] PATH_INFO: `/v3/projects/5faba0985c464030a7b238905671c767/users/ed55d058f3f042709386d5742c3c6b7a/roles/8e2919e79df040e8926f21607818b58c` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:23:30.159351 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99565bcd-5a92-4163-9d0d-177a188499ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=5faba0985c464030a7b238905671c767, user_id=ed55d058f3f042709386d5742c3c6b7a, role_id=8e2919e79df040e8926f21607818b58c)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:23:30.162593 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99565bcd-5a92-4163-9d0d-177a188499ff admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:23:30.326590 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1206/2387] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Sep 23 15:23:30 2022] PUT /identity/v3/projects/5faba0985c464030a7b238905671c767/users/ed55d058f3f042709386d5742c3c6b7a/roles/8e2919e79df040e8926f21607818b58c => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:23:30.334506 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54287ea0-ed4c-4c02-9007-8a0e8e3b9200 admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:23:30.337245 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:23:30.347270 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54287ea0-ed4c-4c02-9007-8a0e8e3b9200 admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:23:30.441993 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5aec49-5ea1-4cd5-9ade-9857d35d830d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:23:30.442774 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5aec49-5ea1-4cd5-9ade-9857d35d830d admin admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:23:30.443054 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5aec49-5ea1-4cd5-9ade-9857d35d830d admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:23:30.443329 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5aec49-5ea1-4cd5-9ade-9857d35d830d admin admin] PATH_INFO: `/v3/roles` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:23:30.444004 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5aec49-5ea1-4cd5-9ade-9857d35d830d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:23:30.445626 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5aec49-5ea1-4cd5-9ade-9857d35d830d admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:23:30.468597 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1182/2388] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Sep 23 15:23:30 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:23:30.473915 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99565bcd-5a92-4163-9d0d-177a188499ff admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:23:30.494312 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99565bcd-5a92-4163-9d0d-177a188499ff admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:23:30.500759 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57b7fda-5074-4849-b85f-577d72ddcc9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:23:30.503547 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57b7fda-5074-4849-b85f-577d72ddcc9b admin admin] REQUEST_METHOD: `PUT` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:23:30.503547 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57b7fda-5074-4849-b85f-577d72ddcc9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:23:30.503547 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57b7fda-5074-4849-b85f-577d72ddcc9b admin admin] PATH_INFO: `/v3/projects/5faba0985c464030a7b238905671c767/users/ed55d058f3f042709386d5742c3c6b7a/roles/8e2919e79df040e8926f21607818b58c` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:23:30.504696 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57b7fda-5074-4849-b85f-577d72ddcc9b admin admin] RBAC: Authorizing `identity:create_grant(project_id=5faba0985c464030a7b238905671c767, user_id=ed55d058f3f042709386d5742c3c6b7a, role_id=8e2919e79df040e8926f21607818b58c)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:23:30.508155 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57b7fda-5074-4849-b85f-577d72ddcc9b admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:23:30.522857 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1207/2389] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Sep 23 15:23:30 2022] PUT /identity/v3/projects/5faba0985c464030a7b238905671c767/users/ed55d058f3f042709386d5742c3c6b7a/roles/8e2919e79df040e8926f21607818b58c => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:23:31.695942 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:23:32.159081 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=84,keystone:INSERT=4,keystone:DELETE=12 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:23:33.604570 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:23:33.844612 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:23:34.596244 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:23:37.654607 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444737e0-ae6a-44b2-ae73-214ae9555ea7 None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:23:37.655581 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444737e0-ae6a-44b2-ae73-214ae9555ea7 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:23:37.655581 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444737e0-ae6a-44b2-ae73-214ae9555ea7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:23:37.685038 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-444737e0-ae6a-44b2-ae73-214ae9555ea7 None None] MFA Rules not processed for user `ed55d058f3f042709386d5742c3c6b7a`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:23:37.686252 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-444737e0-ae6a-44b2-ae73-214ae9555ea7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:23:37.699477 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:23:37.756296 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1183/2390] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:23:37 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 15:23:37.769152 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57b7fda-5074-4849-b85f-577d72ddcc9b admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:23:37.770963 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-d57b7fda-5074-4849-b85f-577d72ddcc9b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:23:37.832759 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57b7fda-5074-4849-b85f-577d72ddcc9b admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:23:37.836426 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69554d7d-6dca-4ec2-a8c6-7bff1f8341a9 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51c39f1d05634589a88c0b6477bd014e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:23:37.837062 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69554d7d-6dca-4ec2-a8c6-7bff1f8341a9 service glance] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:23:37.837343 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69554d7d-6dca-4ec2-a8c6-7bff1f8341a9 service glance] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:23:37.837612 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69554d7d-6dca-4ec2-a8c6-7bff1f8341a9 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:23:37.899420 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69554d7d-6dca-4ec2-a8c6-7bff1f8341a9 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:23:37.901191 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69554d7d-6dca-4ec2-a8c6-7bff1f8341a9 service glance] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:23:37.911110 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1208/2391] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:23:37 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:23:38.430127 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee704a7-6f40-4901-a27b-51e17078eb98 None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:23:38.430428 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee704a7-6f40-4901-a27b-51e17078eb98 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:23:38.430696 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee704a7-6f40-4901-a27b-51e17078eb98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:23:38.462756 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-fee704a7-6f40-4901-a27b-51e17078eb98 None None] MFA Rules not processed for user `68206a6b4f244d45bf7bf1504396e430`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:23:38.463970 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-fee704a7-6f40-4901-a27b-51e17078eb98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:23:38.522011 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1184/2392] 10.222.0.55 () {58 vars in 1009 bytes} [Fri Sep 23 15:23:38 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:23:38.537260 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69554d7d-6dca-4ec2-a8c6-7bff1f8341a9 service glance] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:23:38.592594 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:23:38.605306 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69554d7d-6dca-4ec2-a8c6-7bff1f8341a9 service glance] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:23:38.610636 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5265a4-61db-4cf4-b8cc-faa2e7321c4d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2228c0b49a0249bbaf524a8c4e9e108b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:23:38.611241 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5265a4-61db-4cf4-b8cc-faa2e7321c4d service swift] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:23:38.611519 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5265a4-61db-4cf4-b8cc-faa2e7321c4d service swift] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:23:38.611789 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5265a4-61db-4cf4-b8cc-faa2e7321c4d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:23:38.619772 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5265a4-61db-4cf4-b8cc-faa2e7321c4d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:23:38.621572 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5265a4-61db-4cf4-b8cc-faa2e7321c4d service swift] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:23:38.630216 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1209/2393] 10.222.0.55 () {60 vars in 1384 bytes} [Fri Sep 23 15:23:38 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 23 15:23:42.171327 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee704a7-6f40-4901-a27b-51e17078eb98 None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:23:42.172962 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-fee704a7-6f40-4901-a27b-51e17078eb98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:23:42.241680 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee704a7-6f40-4901-a27b-51e17078eb98 None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:23:42.246217 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db10da2d-2f21-4a0f-aa0c-21520c1a4aff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd259178570864ffeb936ee10dfec6b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:23:42.246956 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db10da2d-2f21-4a0f-aa0c-21520c1a4aff service neutron] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:23:42.247312 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db10da2d-2f21-4a0f-aa0c-21520c1a4aff service neutron] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:23:42.247663 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db10da2d-2f21-4a0f-aa0c-21520c1a4aff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:23:42.249640 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-db10da2d-2f21-4a0f-aa0c-21520c1a4aff service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:23:42.317398 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db10da2d-2f21-4a0f-aa0c-21520c1a4aff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:23:42.318838 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db10da2d-2f21-4a0f-aa0c-21520c1a4aff service neutron] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:23:42.329549 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1185/2394] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:23:42 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:23:48.811408 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=7 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:23:51.642450 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bf8283-5b61-4e12-b1fc-0af41d1c58b0 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:23:51.643128 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bf8283-5b61-4e12-b1fc-0af41d1c58b0 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:23:51.643128 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bf8283-5b61-4e12-b1fc-0af41d1c58b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:23:51.663211 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-96bf8283-5b61-4e12-b1fc-0af41d1c58b0 None None] MFA Rules not processed for user `deda513f4c6448368cfa79203cf0d8d8`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:23:51.664547 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-96bf8283-5b61-4e12-b1fc-0af41d1c58b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:23:51.672281 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1210/2395] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:23:51 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:23:52.325050 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=97 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:23:58.501931 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d29edeb-76b6-4b83-ac52-1c8dc8c561bf None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:23:58.502547 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d29edeb-76b6-4b83-ac52-1c8dc8c561bf None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:23:58.502547 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d29edeb-76b6-4b83-ac52-1c8dc8c561bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:23:58.525916 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-3d29edeb-76b6-4b83-ac52-1c8dc8c561bf None None] MFA Rules not processed for user `68206a6b4f244d45bf7bf1504396e430`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:23:58.527618 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-3d29edeb-76b6-4b83-ac52-1c8dc8c561bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:23:58.535182 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1186/2396] 10.222.0.55 () {58 vars in 967 bytes} [Fri Sep 23 15:23:58 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:23:58.558813 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bf8283-5b61-4e12-b1fc-0af41d1c58b0 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:23:58.571111 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bf8283-5b61-4e12-b1fc-0af41d1c58b0 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:23:58.576154 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb99d77-5c27-44f4-9478-82477eed2036 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2228c0b49a0249bbaf524a8c4e9e108b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:23:58.577018 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb99d77-5c27-44f4-9478-82477eed2036 service swift] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:23:58.577018 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb99d77-5c27-44f4-9478-82477eed2036 service swift] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:23:58.577371 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb99d77-5c27-44f4-9478-82477eed2036 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:23:58.585654 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb99d77-5c27-44f4-9478-82477eed2036 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:23:58.587232 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb99d77-5c27-44f4-9478-82477eed2036 service swift] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:23:58.595869 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1211/2397] 10.222.0.55 () {60 vars in 1384 bytes} [Fri Sep 23 15:23:58 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 23 15:24:04.184801 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72282219-62f3-4f67-a55e-a958d0252f88 None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:24:04.185289 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72282219-62f3-4f67-a55e-a958d0252f88 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:24:04.185384 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72282219-62f3-4f67-a55e-a958d0252f88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:24:04.224539 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-72282219-62f3-4f67-a55e-a958d0252f88 None None] MFA Rules not processed for user `68206a6b4f244d45bf7bf1504396e430`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:24:04.226959 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-72282219-62f3-4f67-a55e-a958d0252f88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:24:04.238777 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1187/2398] 10.222.0.55 () {58 vars in 1009 bytes} [Fri Sep 23 15:24:04 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:24:04.257867 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb99d77-5c27-44f4-9478-82477eed2036 service swift] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:24:04.270654 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb99d77-5c27-44f4-9478-82477eed2036 service swift] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:24:04.277162 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c91fe6-f6ef-44b9-97cb-3913de85ec73 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2228c0b49a0249bbaf524a8c4e9e108b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:24:04.277445 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c91fe6-f6ef-44b9-97cb-3913de85ec73 service swift] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:24:04.280127 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c91fe6-f6ef-44b9-97cb-3913de85ec73 service swift] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:24:04.280127 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c91fe6-f6ef-44b9-97cb-3913de85ec73 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:24:04.292071 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c91fe6-f6ef-44b9-97cb-3913de85ec73 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:24:04.295676 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c91fe6-f6ef-44b9-97cb-3913de85ec73 service swift] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:24:04.328130 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1212/2399] 10.222.0.55 () {60 vars in 1384 bytes} [Fri Sep 23 15:24:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 23 15:24:14.203495 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=6 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:24:14.305448 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=15 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:24:33.190055 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d5ac04-82f2-475d-990a-442c61337573 None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:24:33.196192 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d5ac04-82f2-475d-990a-442c61337573 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:24:33.196192 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d5ac04-82f2-475d-990a-442c61337573 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:24:33.211351 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-d0d5ac04-82f2-475d-990a-442c61337573 None None] MFA Rules not processed for user `ee65edb3211b4b71a771594a90a342ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:24:33.213300 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-d0d5ac04-82f2-475d-990a-442c61337573 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:24:33.270387 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1188/2400] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:24:33 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:24:36.557359 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f65b33a-4df3-42b5-b55d-5224a8552de8 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:24:36.558215 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f65b33a-4df3-42b5-b55d-5224a8552de8 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:24:36.558487 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f65b33a-4df3-42b5-b55d-5224a8552de8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:24:36.578543 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-2f65b33a-4df3-42b5-b55d-5224a8552de8 None None] MFA Rules not processed for user `a5514abdc8b54282901367a4f2d02856`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:24:36.579802 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-2f65b33a-4df3-42b5-b55d-5224a8552de8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:24:36.638107 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1213/2401] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:24:36 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:24:38.188462 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d5ac04-82f2-475d-990a-442c61337573 None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:24:38.200097 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d5ac04-82f2-475d-990a-442c61337573 None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:24:38.204316 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa4626d-1aab-4ec0-9a01-ed1a60956a89 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd259178570864ffeb936ee10dfec6b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:24:38.205122 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa4626d-1aab-4ec0-9a01-ed1a60956a89 service neutron] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:24:38.205449 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa4626d-1aab-4ec0-9a01-ed1a60956a89 service neutron] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:24:38.205760 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa4626d-1aab-4ec0-9a01-ed1a60956a89 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:24:38.228908 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa4626d-1aab-4ec0-9a01-ed1a60956a89 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:24:38.230599 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa4626d-1aab-4ec0-9a01-ed1a60956a89 service neutron] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:24:38.245749 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1189/2402] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:24:38 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:24:46.630276 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=25 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:24:48.240415 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=31 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:24:59.554083 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f65b33a-4df3-42b5-b55d-5224a8552de8 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:24:59.565873 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f65b33a-4df3-42b5-b55d-5224a8552de8 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:24:59.571384 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2337a182-8711-4f36-a0f3-08cfc27b171b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd259178570864ffeb936ee10dfec6b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:24:59.572338 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2337a182-8711-4f36-a0f3-08cfc27b171b service neutron] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:24:59.572873 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2337a182-8711-4f36-a0f3-08cfc27b171b service neutron] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:24:59.573300 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2337a182-8711-4f36-a0f3-08cfc27b171b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:24:59.575380 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-2337a182-8711-4f36-a0f3-08cfc27b171b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:24:59.664956 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2337a182-8711-4f36-a0f3-08cfc27b171b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:24:59.666419 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2337a182-8711-4f36-a0f3-08cfc27b171b service neutron] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:24:59.693389 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1214/2403] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:24:59 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:25:04.827970 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed24e6a-7941-4125-ad10-b10baf388cf1 None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:25:04.828560 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed24e6a-7941-4125-ad10-b10baf388cf1 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:25:04.828560 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed24e6a-7941-4125-ad10-b10baf388cf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:25:04.849076 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-3ed24e6a-7941-4125-ad10-b10baf388cf1 None None] MFA Rules not processed for user `42dc378333ae4bbdbd42ce041ba15d3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:25:04.850274 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-3ed24e6a-7941-4125-ad10-b10baf388cf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:25:04.908769 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1190/2404] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:25:04 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 15:25:05.866234 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2337a182-8711-4f36-a0f3-08cfc27b171b service neutron] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:25:05.869216 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-2337a182-8711-4f36-a0f3-08cfc27b171b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:25:05.881904 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2337a182-8711-4f36-a0f3-08cfc27b171b service neutron] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:25:05.887030 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74353680-2248-4382-adf0-3eea3ee7150d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd259178570864ffeb936ee10dfec6b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:25:05.887648 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74353680-2248-4382-adf0-3eea3ee7150d service neutron] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:25:05.887933 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74353680-2248-4382-adf0-3eea3ee7150d service neutron] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:25:05.888240 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74353680-2248-4382-adf0-3eea3ee7150d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:25:05.890205 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-74353680-2248-4382-adf0-3eea3ee7150d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:25:05.910757 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74353680-2248-4382-adf0-3eea3ee7150d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:25:05.912390 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74353680-2248-4382-adf0-3eea3ee7150d service neutron] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:25:05.925596 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1215/2405] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:25:05 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:25:14.901773 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=25 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:25:15.920077 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=40 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:25:19.901158 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed24e6a-7941-4125-ad10-b10baf388cf1 None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:25:19.902650 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-3ed24e6a-7941-4125-ad10-b10baf388cf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:25:19.919910 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed24e6a-7941-4125-ad10-b10baf388cf1 None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:25:19.925550 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7dc97d-1e63-4d6b-a8dc-39eec6de06e3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3c6c7a1325a47be924268e1aa1a7a18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:25:19.926101 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7dc97d-1e63-4d6b-a8dc-39eec6de06e3 service nova] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:25:19.926379 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7dc97d-1e63-4d6b-a8dc-39eec6de06e3 service nova] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:25:19.926653 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7dc97d-1e63-4d6b-a8dc-39eec6de06e3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:25:19.928434 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-9c7dc97d-1e63-4d6b-a8dc-39eec6de06e3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:25:19.940024 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7dc97d-1e63-4d6b-a8dc-39eec6de06e3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:25:19.946331 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7dc97d-1e63-4d6b-a8dc-39eec6de06e3 service nova] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:25:19.951832 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1191/2406] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:25:19 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:25:23.391485 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74353680-2248-4382-adf0-3eea3ee7150d service neutron] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:25:23.394296 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-74353680-2248-4382-adf0-3eea3ee7150d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:25:23.453655 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74353680-2248-4382-adf0-3eea3ee7150d service neutron] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:25:23.457320 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca92987-ff5c-4d9e-aa6c-97f21f1f4c1b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feca6278be904ef8b0dc580a67e477f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:25:23.457926 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca92987-ff5c-4d9e-aa6c-97f21f1f4c1b service cinder] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:25:23.458201 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca92987-ff5c-4d9e-aa6c-97f21f1f4c1b service cinder] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:25:23.458470 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca92987-ff5c-4d9e-aa6c-97f21f1f4c1b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:25:23.460205 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-0ca92987-ff5c-4d9e-aa6c-97f21f1f4c1b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:25:23.471467 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca92987-ff5c-4d9e-aa6c-97f21f1f4c1b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:25:23.473066 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca92987-ff5c-4d9e-aa6c-97f21f1f4c1b service cinder] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:25:23.483094 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1216/2407] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:25:23 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 23 15:25:23.931775 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7dc97d-1e63-4d6b-a8dc-39eec6de06e3 service nova] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:25:23.947796 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7dc97d-1e63-4d6b-a8dc-39eec6de06e3 service nova] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:25:23.957598 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0fc4cc-3f45-415d-9746-c698e9d0bc10 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feca6278be904ef8b0dc580a67e477f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:25:23.958533 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0fc4cc-3f45-415d-9746-c698e9d0bc10 service cinder] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:25:23.959004 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0fc4cc-3f45-415d-9746-c698e9d0bc10 service cinder] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:25:23.959480 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0fc4cc-3f45-415d-9746-c698e9d0bc10 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:25:23.962556 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-ec0fc4cc-3f45-415d-9746-c698e9d0bc10 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:25:24.037575 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0fc4cc-3f45-415d-9746-c698e9d0bc10 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:25:24.038988 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0fc4cc-3f45-415d-9746-c698e9d0bc10 service cinder] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:25:24.050823 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1192/2408] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:25:23 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:25:29.394206 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca92987-ff5c-4d9e-aa6c-97f21f1f4c1b service cinder] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:25:29.405225 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca92987-ff5c-4d9e-aa6c-97f21f1f4c1b service cinder] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:25:29.409025 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d30f12-588f-4785-875a-b2f877b480e7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd259178570864ffeb936ee10dfec6b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:25:29.409633 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d30f12-588f-4785-875a-b2f877b480e7 service neutron] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:25:29.409913 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d30f12-588f-4785-875a-b2f877b480e7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:25:29.410188 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d30f12-588f-4785-875a-b2f877b480e7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:25:29.412188 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-91d30f12-588f-4785-875a-b2f877b480e7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:25:29.423741 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d30f12-588f-4785-875a-b2f877b480e7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:25:29.429039 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d30f12-588f-4785-875a-b2f877b480e7 service neutron] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:25:29.434656 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1217/2409] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:25:29 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:25:34.046874 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=34 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:25:39.430023 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=32 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:25:40.804783 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4eee5c-1c8e-40be-85da-f3273b921398 None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:25:40.805309 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4eee5c-1c8e-40be-85da-f3273b921398 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:25:40.805410 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4eee5c-1c8e-40be-85da-f3273b921398 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:25:40.826649 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-cb4eee5c-1c8e-40be-85da-f3273b921398 None None] MFA Rules not processed for user `f5f0ecaf38e74c96bc439fde75d93d37`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:25:40.828143 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-cb4eee5c-1c8e-40be-85da-f3273b921398 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:25:40.841897 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1193/2410] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:25:40 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:25:41.393081 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d30f12-588f-4785-875a-b2f877b480e7 service neutron] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:25:41.402770 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d30f12-588f-4785-875a-b2f877b480e7 service neutron] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:25:41.406604 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c050062d-48fa-46c7-aa97-f0fe7f7f72ff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd259178570864ffeb936ee10dfec6b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:25:41.407200 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c050062d-48fa-46c7-aa97-f0fe7f7f72ff service neutron] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:25:41.407475 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c050062d-48fa-46c7-aa97-f0fe7f7f72ff service neutron] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:25:41.407747 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c050062d-48fa-46c7-aa97-f0fe7f7f72ff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:25:41.409646 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-c050062d-48fa-46c7-aa97-f0fe7f7f72ff service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:25:41.419830 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c050062d-48fa-46c7-aa97-f0fe7f7f72ff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:25:41.421268 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c050062d-48fa-46c7-aa97-f0fe7f7f72ff service neutron] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:25:41.429938 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1218/2411] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:25:41 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:25:41.851941 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4eee5c-1c8e-40be-85da-f3273b921398 None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:25:41.853419 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-cb4eee5c-1c8e-40be-85da-f3273b921398 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:25:41.946789 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4eee5c-1c8e-40be-85da-f3273b921398 None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:25:41.951917 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e416cee-ff50-4fbf-a50c-9ec48c623173 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09f1a55d180642c781a6e0b017d9f0aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:25:41.952554 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e416cee-ff50-4fbf-a50c-9ec48c623173 service placement] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:25:41.952828 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e416cee-ff50-4fbf-a50c-9ec48c623173 service placement] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:25:41.953099 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e416cee-ff50-4fbf-a50c-9ec48c623173 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:25:41.954847 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-3e416cee-ff50-4fbf-a50c-9ec48c623173 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:25:41.968047 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e416cee-ff50-4fbf-a50c-9ec48c623173 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:25:41.969741 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e416cee-ff50-4fbf-a50c-9ec48c623173 service placement] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:25:41.981560 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1194/2412] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:25:41 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:25:48.044294 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c050062d-48fa-46c7-aa97-f0fe7f7f72ff service neutron] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:25:48.059076 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c050062d-48fa-46c7-aa97-f0fe7f7f72ff service neutron] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:25:48.066984 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41713293-d1b8-4455-af82-35bae8c70c09 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3c6c7a1325a47be924268e1aa1a7a18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:25:48.067700 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41713293-d1b8-4455-af82-35bae8c70c09 service nova] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:25:48.068207 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41713293-d1b8-4455-af82-35bae8c70c09 service nova] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:25:48.068538 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41713293-d1b8-4455-af82-35bae8c70c09 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:25:48.070390 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-41713293-d1b8-4455-af82-35bae8c70c09 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:25:48.084257 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41713293-d1b8-4455-af82-35bae8c70c09 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:25:48.085801 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41713293-d1b8-4455-af82-35bae8c70c09 service nova] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:25:48.097052 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1219/2413] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:25:48 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 23 15:25:51.976155 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=35 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:25:51.999837 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e416cee-ff50-4fbf-a50c-9ec48c623173 service placement] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:25:52.001480 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-3e416cee-ff50-4fbf-a50c-9ec48c623173 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:25:52.014749 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e416cee-ff50-4fbf-a50c-9ec48c623173 service placement] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:25:52.020250 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cc5f39-b3cd-4e37-9fe5-bd15cc27abae service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09f1a55d180642c781a6e0b017d9f0aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:25:52.020930 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cc5f39-b3cd-4e37-9fe5-bd15cc27abae service placement] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:25:52.021380 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cc5f39-b3cd-4e37-9fe5-bd15cc27abae service placement] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:25:52.021806 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cc5f39-b3cd-4e37-9fe5-bd15cc27abae service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:25:52.025712 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-f5cc5f39-b3cd-4e37-9fe5-bd15cc27abae service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:25:52.039112 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cc5f39-b3cd-4e37-9fe5-bd15cc27abae service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:25:52.040905 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cc5f39-b3cd-4e37-9fe5-bd15cc27abae service placement] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:25:52.054298 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1195/2414] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:25:51 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:25:52.175713 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41713293-d1b8-4455-af82-35bae8c70c09 service nova] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:25:52.178090 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-41713293-d1b8-4455-af82-35bae8c70c09 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:25:52.251446 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41713293-d1b8-4455-af82-35bae8c70c09 service nova] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:25:52.255931 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477663fe-75a0-4466-a19b-3f5edb058a13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:25:52.256590 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477663fe-75a0-4466-a19b-3f5edb058a13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:25:52.256872 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477663fe-75a0-4466-a19b-3f5edb058a13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:25:52.257150 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477663fe-75a0-4466-a19b-3f5edb058a13 admin admin] PATH_INFO: `/v3/users/ee65edb3211b4b71a771594a90a342ff` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:25:52.258277 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477663fe-75a0-4466-a19b-3f5edb058a13 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ee65edb3211b4b71a771594a90a342ff)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:25:52.260266 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477663fe-75a0-4466-a19b-3f5edb058a13 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:25:52.485131 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-477663fe-75a0-4466-a19b-3f5edb058a13 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ee65edb3211b4b71a771594a90a342ff'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:25:52.497500 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-477663fe-75a0-4466-a19b-3f5edb058a13 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ee65edb3211b4b71a771594a90a342ff'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:25:52.510274 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-477663fe-75a0-4466-a19b-3f5edb058a13 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ee65edb3211b4b71a771594a90a342ff'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:25:52.594123 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-477663fe-75a0-4466-a19b-3f5edb058a13 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ee65edb3211b4b71a771594a90a342ff'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:25:52.594788 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-477663fe-75a0-4466-a19b-3f5edb058a13 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ee65edb3211b4b71a771594a90a342ff'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:25:52.601227 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1220/2415] 10.222.0.55 () {58 vars in 1224 bytes} [Fri Sep 23 15:25:52 2022] DELETE /identity/v3/users/ee65edb3211b4b71a771594a90a342ff => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:25:52.904139 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cc5f39-b3cd-4e37-9fe5-bd15cc27abae service placement] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:25:52.915460 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cc5f39-b3cd-4e37-9fe5-bd15cc27abae service placement] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:25:52.982421 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268c994d-bb0b-4e5d-935d-8159789609cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:25:52.983005 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268c994d-bb0b-4e5d-935d-8159789609cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:25:52.983105 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268c994d-bb0b-4e5d-935d-8159789609cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:25:52.983371 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268c994d-bb0b-4e5d-935d-8159789609cd admin admin] PATH_INFO: `/v3/projects/c9488f4d4af441168341205e0e936ce4` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:25:52.986419 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268c994d-bb0b-4e5d-935d-8159789609cd admin admin] RBAC: Authorizing `identity:delete_project(project_id=c9488f4d4af441168341205e0e936ce4)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:25:52.988706 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268c994d-bb0b-4e5d-935d-8159789609cd admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:25:53.113240 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-268c994d-bb0b-4e5d-935d-8159789609cd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c9488f4d4af441168341205e0e936ce4', 'request_id': 'req-268c994d-bb0b-4e5d-935d-8159789609cd'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:25:53.118786 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-268c994d-bb0b-4e5d-935d-8159789609cd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c9488f4d4af441168341205e0e936ce4', 'request_id': 'req-268c994d-bb0b-4e5d-935d-8159789609cd'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:25:53.248901 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-268c994d-bb0b-4e5d-935d-8159789609cd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c9488f4d4af441168341205e0e936ce4', 'request_id': 'req-268c994d-bb0b-4e5d-935d-8159789609cd'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:25:53.262513 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-268c994d-bb0b-4e5d-935d-8159789609cd admin admin] The token cache is being invalidate because project c9488f4d4af441168341205e0e936ce4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81993) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 15:25:53.262997 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-268c994d-bb0b-4e5d-935d-8159789609cd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:25:53.263711 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-268c994d-bb0b-4e5d-935d-8159789609cd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:25:53.265499 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1196/2416] 10.222.0.55 () {58 vars in 1233 bytes} [Fri Sep 23 15:25:52 2022] DELETE /identity/v3/projects/c9488f4d4af441168341205e0e936ce4 => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:25:53.270479 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0debe08-85e4-4de3-bc34-f854fc3236c3 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:25:53.270984 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0debe08-85e4-4de3-bc34-f854fc3236c3 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:25:53.271070 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0debe08-85e4-4de3-bc34-f854fc3236c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:25:53.294307 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-a0debe08-85e4-4de3-bc34-f854fc3236c3 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:25:53.295634 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-a0debe08-85e4-4de3-bc34-f854fc3236c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:25:53.349556 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:25:53.363147 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1221/2417] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:25:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 15:25:53.370986 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75621f66-8f44-441c-b84e-4c754e1be9ba None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:25:53.371270 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75621f66-8f44-441c-b84e-4c754e1be9ba None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:25:53.371542 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75621f66-8f44-441c-b84e-4c754e1be9ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:25:53.392754 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-75621f66-8f44-441c-b84e-4c754e1be9ba None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:25:53.393956 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-75621f66-8f44-441c-b84e-4c754e1be9ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:25:53.401953 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1197/2418] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:25:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:25:53.406817 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0debe08-85e4-4de3-bc34-f854fc3236c3 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:25:53.416333 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0debe08-85e4-4de3-bc34-f854fc3236c3 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:25:53.421872 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb83c4af-04ba-485f-a97f-0ecae9652991 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:25:53.422544 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb83c4af-04ba-485f-a97f-0ecae9652991 admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:25:53.422825 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb83c4af-04ba-485f-a97f-0ecae9652991 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:25:53.423101 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb83c4af-04ba-485f-a97f-0ecae9652991 admin admin] PATH_INFO: `/v3/domains` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:25:53.423794 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb83c4af-04ba-485f-a97f-0ecae9652991 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:25:53.424147 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb83c4af-04ba-485f-a97f-0ecae9652991 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:25:53.425744 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb83c4af-04ba-485f-a97f-0ecae9652991 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:25:53.437979 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1222/2419] 10.222.0.55 () {58 vars in 1153 bytes} [Fri Sep 23 15:25:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:25:53.442656 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75621f66-8f44-441c-b84e-4c754e1be9ba None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:25:53.453838 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75621f66-8f44-441c-b84e-4c754e1be9ba None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:25:53.459013 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfb8082-0fdd-4f29-9fa6-fa2f6b1d50c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:25:53.459628 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfb8082-0fdd-4f29-9fa6-fa2f6b1d50c9 admin admin] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:25:53.459908 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfb8082-0fdd-4f29-9fa6-fa2f6b1d50c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:25:53.460213 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfb8082-0fdd-4f29-9fa6-fa2f6b1d50c9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:25:53.460976 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfb8082-0fdd-4f29-9fa6-fa2f6b1d50c9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:25:53.464306 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfb8082-0fdd-4f29-9fa6-fa2f6b1d50c9 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:25:53.574285 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1198/2420] 10.222.0.55 () {60 vars in 1153 bytes} [Fri Sep 23 15:25:53 2022] POST /identity/v3/projects => generated 399 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 23 15:25:53.575567 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb83c4af-04ba-485f-a97f-0ecae9652991 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:25:53.586239 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb83c4af-04ba-485f-a97f-0ecae9652991 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:25:53.646144 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6898b3-e138-497b-81d8-08314e967501 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:25:53.648902 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6898b3-e138-497b-81d8-08314e967501 admin admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:25:53.648902 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6898b3-e138-497b-81d8-08314e967501 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:25:53.648902 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6898b3-e138-497b-81d8-08314e967501 admin admin] PATH_INFO: `/v3/users` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:25:53.648902 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6898b3-e138-497b-81d8-08314e967501 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:25:53.650247 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6898b3-e138-497b-81d8-08314e967501 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:25:53.823714 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1223/2421] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Sep 23 15:25:53 2022] POST /identity/v3/users => generated 300 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:25:53.872600 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfb8082-0fdd-4f29-9fa6-fa2f6b1d50c9 admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:25:53.872600 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfb8082-0fdd-4f29-9fa6-fa2f6b1d50c9 admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:25:53.872600 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc259ed4-a40f-4161-8aa2-3d7c1295fb14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:25:53.872600 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc259ed4-a40f-4161-8aa2-3d7c1295fb14 admin admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:25:53.873549 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc259ed4-a40f-4161-8aa2-3d7c1295fb14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:25:53.873549 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc259ed4-a40f-4161-8aa2-3d7c1295fb14 admin admin] PATH_INFO: `/v3/roles` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:25:53.873549 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc259ed4-a40f-4161-8aa2-3d7c1295fb14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:25:53.873549 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc259ed4-a40f-4161-8aa2-3d7c1295fb14 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:25:53.873549 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1199/2422] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Sep 23 15:25:53 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:25:53.874269 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6898b3-e138-497b-81d8-08314e967501 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:25:53.874269 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6898b3-e138-497b-81d8-08314e967501 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:25:53.878769 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e0145c-d4a2-4856-b4e3-310a5b1ef671 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:25:53.879456 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e0145c-d4a2-4856-b4e3-310a5b1ef671 admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:25:53.879820 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e0145c-d4a2-4856-b4e3-310a5b1ef671 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:25:53.880200 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e0145c-d4a2-4856-b4e3-310a5b1ef671 admin admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:25:53.880921 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e0145c-d4a2-4856-b4e3-310a5b1ef671 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:25:53.882702 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e0145c-d4a2-4856-b4e3-310a5b1ef671 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:25:53.896816 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1224/2423] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Sep 23 15:25:53 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:25:53.901148 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc259ed4-a40f-4161-8aa2-3d7c1295fb14 admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:25:53.912245 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc259ed4-a40f-4161-8aa2-3d7c1295fb14 admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:25:53.916876 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59e202a-b33b-4af0-8520-3d8b4db5c32a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:25:53.922190 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59e202a-b33b-4af0-8520-3d8b4db5c32a admin admin] REQUEST_METHOD: `PUT` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:25:53.922190 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59e202a-b33b-4af0-8520-3d8b4db5c32a admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:25:53.922190 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59e202a-b33b-4af0-8520-3d8b4db5c32a admin admin] PATH_INFO: `/v3/projects/0d1843be4df843d6a5ae7118346a1b09/users/6efc93f1a5c54ccd9f95b676c7a0716a/roles/af257be2dee54d6b942f58650196582c` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:25:53.935518 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59e202a-b33b-4af0-8520-3d8b4db5c32a admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d1843be4df843d6a5ae7118346a1b09, user_id=6efc93f1a5c54ccd9f95b676c7a0716a, role_id=af257be2dee54d6b942f58650196582c)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:25:53.938364 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59e202a-b33b-4af0-8520-3d8b4db5c32a admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:25:54.041114 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1200/2424] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Sep 23 15:25:53 2022] PUT /identity/v3/projects/0d1843be4df843d6a5ae7118346a1b09/users/6efc93f1a5c54ccd9f95b676c7a0716a/roles/af257be2dee54d6b942f58650196582c => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 15:25:54.110013 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a42dea-1cc1-4bae-b4b5-3b5c4537a2d3 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:25:54.110327 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a42dea-1cc1-4bae-b4b5-3b5c4537a2d3 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:25:54.113534 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a42dea-1cc1-4bae-b4b5-3b5c4537a2d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:25:54.144299 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-74a42dea-1cc1-4bae-b4b5-3b5c4537a2d3 None None] MFA Rules not processed for user `6efc93f1a5c54ccd9f95b676c7a0716a`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:25:54.148039 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-74a42dea-1cc1-4bae-b4b5-3b5c4537a2d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:25:54.215544 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1225/2425] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:25:54 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:25:54.228631 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59e202a-b33b-4af0-8520-3d8b4db5c32a admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:25:54.230430 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-e59e202a-b33b-4af0-8520-3d8b4db5c32a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:25:54.248973 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:25:54.298691 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59e202a-b33b-4af0-8520-3d8b4db5c32a admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:25:54.303250 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e555ac67-ce05-47e7-99aa-a5d2ee423ec1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feca6278be904ef8b0dc580a67e477f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:25:54.303883 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e555ac67-ce05-47e7-99aa-a5d2ee423ec1 service cinder] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:25:54.304194 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e555ac67-ce05-47e7-99aa-a5d2ee423ec1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:25:54.304467 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e555ac67-ce05-47e7-99aa-a5d2ee423ec1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:25:54.314054 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e555ac67-ce05-47e7-99aa-a5d2ee423ec1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:25:54.315497 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e555ac67-ce05-47e7-99aa-a5d2ee423ec1 service cinder] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:25:54.327072 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1201/2426] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:25:54 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:25:56.588766 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a42dea-1cc1-4bae-b4b5-3b5c4537a2d3 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:25:56.590343 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-74a42dea-1cc1-4bae-b4b5-3b5c4537a2d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:25:56.641623 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:25:56.667411 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a42dea-1cc1-4bae-b4b5-3b5c4537a2d3 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:25:56.674552 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ebf793-9149-413d-b015-16e4fe6f60ed service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3c6c7a1325a47be924268e1aa1a7a18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:25:56.675175 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ebf793-9149-413d-b015-16e4fe6f60ed service nova] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:25:56.675459 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ebf793-9149-413d-b015-16e4fe6f60ed service nova] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:25:56.675788 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ebf793-9149-413d-b015-16e4fe6f60ed service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:25:56.677595 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-28ebf793-9149-413d-b015-16e4fe6f60ed service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:25:56.690229 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ebf793-9149-413d-b015-16e4fe6f60ed service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:25:56.691798 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ebf793-9149-413d-b015-16e4fe6f60ed service nova] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:25:56.702898 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1226/2427] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:25:56 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:25:56.792914 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e555ac67-ce05-47e7-99aa-a5d2ee423ec1 service cinder] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:25:56.794575 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-e555ac67-ce05-47e7-99aa-a5d2ee423ec1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:25:56.856920 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e555ac67-ce05-47e7-99aa-a5d2ee423ec1 service cinder] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:25:56.861212 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9557d1c2-a5f2-4224-ada1-3f04ca25ed9d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd259178570864ffeb936ee10dfec6b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:25:56.861823 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9557d1c2-a5f2-4224-ada1-3f04ca25ed9d service neutron] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:25:56.862103 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9557d1c2-a5f2-4224-ada1-3f04ca25ed9d service neutron] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:25:56.862376 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9557d1c2-a5f2-4224-ada1-3f04ca25ed9d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:25:56.864216 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-9557d1c2-a5f2-4224-ada1-3f04ca25ed9d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:25:56.937321 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9557d1c2-a5f2-4224-ada1-3f04ca25ed9d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:25:56.939143 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9557d1c2-a5f2-4224-ada1-3f04ca25ed9d service neutron] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:25:56.949511 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1202/2428] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:25:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:26:01.257310 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ebf793-9149-413d-b015-16e4fe6f60ed service nova] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:26:01.258781 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-28ebf793-9149-413d-b015-16e4fe6f60ed service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:26:01.352642 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ebf793-9149-413d-b015-16e4fe6f60ed service nova] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:26:01.357765 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284a6007-33cf-44a6-bfa0-f066f9faab8c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09f1a55d180642c781a6e0b017d9f0aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:26:01.358975 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284a6007-33cf-44a6-bfa0-f066f9faab8c service placement] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:01.359216 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284a6007-33cf-44a6-bfa0-f066f9faab8c service placement] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:01.359485 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284a6007-33cf-44a6-bfa0-f066f9faab8c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:01.361266 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-284a6007-33cf-44a6-bfa0-f066f9faab8c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:26:01.379660 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284a6007-33cf-44a6-bfa0-f066f9faab8c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:26:01.381325 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284a6007-33cf-44a6-bfa0-f066f9faab8c service placement] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:26:01.392563 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1227/2429] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:26:01 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:26:01.420696 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9557d1c2-a5f2-4224-ada1-3f04ca25ed9d service neutron] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:26:01.422727 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-9557d1c2-a5f2-4224-ada1-3f04ca25ed9d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:26:01.437028 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9557d1c2-a5f2-4224-ada1-3f04ca25ed9d service neutron] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:26:01.442046 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d993d8-0ed1-453f-873e-713cf816337d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:26:01.442846 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d993d8-0ed1-453f-873e-713cf816337d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:01.443120 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d993d8-0ed1-453f-873e-713cf816337d admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:01.450529 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d993d8-0ed1-453f-873e-713cf816337d admin admin] PATH_INFO: `/v3/users/a5514abdc8b54282901367a4f2d02856` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:01.450529 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d993d8-0ed1-453f-873e-713cf816337d admin admin] RBAC: Authorizing `identity:delete_user(user_id=a5514abdc8b54282901367a4f2d02856)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:26:01.450529 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d993d8-0ed1-453f-873e-713cf816337d admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:26:01.735148 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-d5d993d8-0ed1-453f-873e-713cf816337d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a5514abdc8b54282901367a4f2d02856'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:01.746378 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:26:01.748985 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-d5d993d8-0ed1-453f-873e-713cf816337d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a5514abdc8b54282901367a4f2d02856'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:01.758537 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-d5d993d8-0ed1-453f-873e-713cf816337d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a5514abdc8b54282901367a4f2d02856'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:01.939833 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-d5d993d8-0ed1-453f-873e-713cf816337d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a5514abdc8b54282901367a4f2d02856'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:01.940674 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-d5d993d8-0ed1-453f-873e-713cf816337d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a5514abdc8b54282901367a4f2d02856'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:01.947580 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1203/2430] 10.222.0.55 () {58 vars in 1224 bytes} [Fri Sep 23 15:26:01 2022] DELETE /identity/v3/users/a5514abdc8b54282901367a4f2d02856 => generated 0 bytes in 527 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:26:02.281556 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284a6007-33cf-44a6-bfa0-f066f9faab8c service placement] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:26:02.297450 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284a6007-33cf-44a6-bfa0-f066f9faab8c service placement] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:26:02.373738 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6795009b-9c7a-4817-83bf-6ff61cd859bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:26:02.374963 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6795009b-9c7a-4817-83bf-6ff61cd859bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:02.375340 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6795009b-9c7a-4817-83bf-6ff61cd859bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:02.375623 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6795009b-9c7a-4817-83bf-6ff61cd859bc admin admin] PATH_INFO: `/v3/projects/a079aa0c75bf44619342f53006d88ff9` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:02.376845 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6795009b-9c7a-4817-83bf-6ff61cd859bc admin admin] RBAC: Authorizing `identity:delete_project(project_id=a079aa0c75bf44619342f53006d88ff9)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:26:02.382396 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6795009b-9c7a-4817-83bf-6ff61cd859bc admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:26:02.506050 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-6795009b-9c7a-4817-83bf-6ff61cd859bc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a079aa0c75bf44619342f53006d88ff9', 'request_id': 'req-6795009b-9c7a-4817-83bf-6ff61cd859bc'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:02.512818 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-6795009b-9c7a-4817-83bf-6ff61cd859bc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a079aa0c75bf44619342f53006d88ff9', 'request_id': 'req-6795009b-9c7a-4817-83bf-6ff61cd859bc'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:02.520409 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-6795009b-9c7a-4817-83bf-6ff61cd859bc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a079aa0c75bf44619342f53006d88ff9', 'request_id': 'req-6795009b-9c7a-4817-83bf-6ff61cd859bc'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:02.523001 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:26:02.625480 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-6795009b-9c7a-4817-83bf-6ff61cd859bc admin admin] The token cache is being invalidate because project a079aa0c75bf44619342f53006d88ff9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81994) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 15:26:02.625769 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-6795009b-9c7a-4817-83bf-6ff61cd859bc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:02.626406 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-6795009b-9c7a-4817-83bf-6ff61cd859bc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:02.627930 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1228/2431] 10.222.0.55 () {58 vars in 1233 bytes} [Fri Sep 23 15:26:02 2022] DELETE /identity/v3/projects/a079aa0c75bf44619342f53006d88ff9 => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:26:02.632836 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f71c254-ab09-4e99-b435-88b6ed99af2a None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:02.633314 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f71c254-ab09-4e99-b435-88b6ed99af2a None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:02.633411 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f71c254-ab09-4e99-b435-88b6ed99af2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:02.654295 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-6f71c254-ab09-4e99-b435-88b6ed99af2a None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:26:02.656087 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-6f71c254-ab09-4e99-b435-88b6ed99af2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:26:02.718464 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1204/2432] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:26:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 15:26:02.726191 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2ed851-e396-4f54-ba01-65610389cff8 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:02.726481 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2ed851-e396-4f54-ba01-65610389cff8 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:02.726765 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2ed851-e396-4f54-ba01-65610389cff8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:02.748107 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-0c2ed851-e396-4f54-ba01-65610389cff8 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:26:02.750504 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-0c2ed851-e396-4f54-ba01-65610389cff8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:26:02.758689 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1229/2433] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:26:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:26:02.763856 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f71c254-ab09-4e99-b435-88b6ed99af2a None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:26:02.773942 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f71c254-ab09-4e99-b435-88b6ed99af2a None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:26:02.778639 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6745631c-a764-4e04-a9b0-baf94b5fffc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:26:02.779279 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6745631c-a764-4e04-a9b0-baf94b5fffc7 admin admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:02.779564 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6745631c-a764-4e04-a9b0-baf94b5fffc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:02.779838 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6745631c-a764-4e04-a9b0-baf94b5fffc7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:02.780554 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6745631c-a764-4e04-a9b0-baf94b5fffc7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:26:02.781052 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6745631c-a764-4e04-a9b0-baf94b5fffc7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:26:02.782666 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6745631c-a764-4e04-a9b0-baf94b5fffc7 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:26:02.794805 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1205/2434] 10.222.0.55 () {58 vars in 1153 bytes} [Fri Sep 23 15:26:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:26:02.799837 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2ed851-e396-4f54-ba01-65610389cff8 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:26:02.810227 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2ed851-e396-4f54-ba01-65610389cff8 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:26:02.816724 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ee2be6-4f71-4baf-9725-6c30dac15382 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:26:02.817521 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ee2be6-4f71-4baf-9725-6c30dac15382 admin admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:02.817816 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ee2be6-4f71-4baf-9725-6c30dac15382 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:02.818103 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ee2be6-4f71-4baf-9725-6c30dac15382 admin admin] PATH_INFO: `/v3/projects` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:02.819286 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ee2be6-4f71-4baf-9725-6c30dac15382 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:26:02.821538 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ee2be6-4f71-4baf-9725-6c30dac15382 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:26:02.965139 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1230/2435] 10.222.0.55 () {60 vars in 1153 bytes} [Fri Sep 23 15:26:02 2022] POST /identity/v3/projects => generated 371 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:26:02.970211 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6745631c-a764-4e04-a9b0-baf94b5fffc7 admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:26:02.980518 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6745631c-a764-4e04-a9b0-baf94b5fffc7 admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:26:03.040903 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5834ae85-1198-4cce-a3cb-e93080cd2236 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:26:03.041771 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5834ae85-1198-4cce-a3cb-e93080cd2236 admin admin] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:03.042159 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5834ae85-1198-4cce-a3cb-e93080cd2236 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:03.042550 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5834ae85-1198-4cce-a3cb-e93080cd2236 admin admin] PATH_INFO: `/v3/users` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:03.047806 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5834ae85-1198-4cce-a3cb-e93080cd2236 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:26:03.047806 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5834ae85-1198-4cce-a3cb-e93080cd2236 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:26:03.148208 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1206/2436] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Sep 23 15:26:02 2022] POST /identity/v3/users => generated 286 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:26:03.152651 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ee2be6-4f71-4baf-9725-6c30dac15382 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:26:03.164082 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ee2be6-4f71-4baf-9725-6c30dac15382 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:26:03.168769 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221306c8-9e94-4c96-b45b-d6f0ba4904b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:26:03.169550 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221306c8-9e94-4c96-b45b-d6f0ba4904b0 admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:03.169700 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221306c8-9e94-4c96-b45b-d6f0ba4904b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:03.169982 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221306c8-9e94-4c96-b45b-d6f0ba4904b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:03.170645 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221306c8-9e94-4c96-b45b-d6f0ba4904b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:26:03.172961 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221306c8-9e94-4c96-b45b-d6f0ba4904b0 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:26:03.183279 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1231/2437] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Sep 23 15:26:03 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:26:03.187679 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5834ae85-1198-4cce-a3cb-e93080cd2236 admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:26:03.198047 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5834ae85-1198-4cce-a3cb-e93080cd2236 admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:26:03.203945 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e114f46d-5063-4f52-a452-a6871e793779 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:26:03.204584 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e114f46d-5063-4f52-a452-a6871e793779 admin admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:03.204865 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e114f46d-5063-4f52-a452-a6871e793779 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:03.205149 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e114f46d-5063-4f52-a452-a6871e793779 admin admin] PATH_INFO: `/v3/roles` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:03.205792 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e114f46d-5063-4f52-a452-a6871e793779 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:26:03.207552 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e114f46d-5063-4f52-a452-a6871e793779 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:26:03.223171 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1207/2438] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Sep 23 15:26:03 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:26:03.227191 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221306c8-9e94-4c96-b45b-d6f0ba4904b0 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:26:03.238242 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221306c8-9e94-4c96-b45b-d6f0ba4904b0 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:26:03.244795 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bb34c1-74ed-4337-a7b4-57fd66c529fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:26:03.245605 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bb34c1-74ed-4337-a7b4-57fd66c529fd admin admin] REQUEST_METHOD: `PUT` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:03.246078 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bb34c1-74ed-4337-a7b4-57fd66c529fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:03.246482 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bb34c1-74ed-4337-a7b4-57fd66c529fd admin admin] PATH_INFO: `/v3/projects/ccf1290dbdf04673949ac453cf3c8939/users/83cddbd5031f45499dbf8f12a44e63ed/roles/af257be2dee54d6b942f58650196582c` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:03.265972 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67bb34c1-74ed-4337-a7b4-57fd66c529fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=ccf1290dbdf04673949ac453cf3c8939, user_id=83cddbd5031f45499dbf8f12a44e63ed, role_id=af257be2dee54d6b942f58650196582c)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:26:03.268934 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67bb34c1-74ed-4337-a7b4-57fd66c529fd admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:26:03.380948 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1232/2439] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Sep 23 15:26:03 2022] PUT /identity/v3/projects/ccf1290dbdf04673949ac453cf3c8939/users/83cddbd5031f45499dbf8f12a44e63ed/roles/af257be2dee54d6b942f58650196582c => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 15:26:03.392132 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31ae3be-cbc5-4cae-af44-056b0db15261 None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:03.392601 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31ae3be-cbc5-4cae-af44-056b0db15261 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:03.392879 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31ae3be-cbc5-4cae-af44-056b0db15261 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:03.426471 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-a31ae3be-cbc5-4cae-af44-056b0db15261 None None] MFA Rules not processed for user `83cddbd5031f45499dbf8f12a44e63ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:26:03.427790 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-a31ae3be-cbc5-4cae-af44-056b0db15261 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:26:03.478306 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:26:03.537413 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1208/2440] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:26:03 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:26:03.548498 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bb34c1-74ed-4337-a7b4-57fd66c529fd admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:26:03.550221 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-67bb34c1-74ed-4337-a7b4-57fd66c529fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:26:03.629809 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bb34c1-74ed-4337-a7b4-57fd66c529fd admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:26:03.635491 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb17c07-0f31-49d5-ae85-78733bd7fd66 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3c6c7a1325a47be924268e1aa1a7a18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:26:03.636007 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb17c07-0f31-49d5-ae85-78733bd7fd66 service nova] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:03.636294 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb17c07-0f31-49d5-ae85-78733bd7fd66 service nova] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:03.636571 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb17c07-0f31-49d5-ae85-78733bd7fd66 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:03.646136 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb17c07-0f31-49d5-ae85-78733bd7fd66 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:26:03.647557 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb17c07-0f31-49d5-ae85-78733bd7fd66 service nova] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:26:03.657857 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1233/2441] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:26:03 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 23 15:26:03.822739 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31ae3be-cbc5-4cae-af44-056b0db15261 None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:26:03.829708 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-a31ae3be-cbc5-4cae-af44-056b0db15261 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:26:03.859172 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31ae3be-cbc5-4cae-af44-056b0db15261 None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:26:03.866548 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1530cdad-1601-4c2d-8d50-72e1260482ab service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3c6c7a1325a47be924268e1aa1a7a18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:26:03.867266 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1530cdad-1601-4c2d-8d50-72e1260482ab service nova] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:03.867556 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1530cdad-1601-4c2d-8d50-72e1260482ab service nova] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:03.868191 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1530cdad-1601-4c2d-8d50-72e1260482ab service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:03.870168 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-1530cdad-1601-4c2d-8d50-72e1260482ab service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:26:03.888792 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1530cdad-1601-4c2d-8d50-72e1260482ab service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:26:03.893464 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1530cdad-1601-4c2d-8d50-72e1260482ab service nova] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:26:03.907725 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1209/2442] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:26:03 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:26:04.304961 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb17c07-0f31-49d5-ae85-78733bd7fd66 service nova] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:26:04.306505 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-9fb17c07-0f31-49d5-ae85-78733bd7fd66 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:26:04.458055 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb17c07-0f31-49d5-ae85-78733bd7fd66 service nova] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:26:04.473076 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f399a08a-5c20-4992-898d-50f214cb4b5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:26:04.476017 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f399a08a-5c20-4992-898d-50f214cb4b5c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:04.476534 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f399a08a-5c20-4992-898d-50f214cb4b5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:04.478499 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f399a08a-5c20-4992-898d-50f214cb4b5c admin admin] PATH_INFO: `/v3/users/6efc93f1a5c54ccd9f95b676c7a0716a` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:04.480819 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f399a08a-5c20-4992-898d-50f214cb4b5c admin admin] RBAC: Authorizing `identity:delete_user(user_id=6efc93f1a5c54ccd9f95b676c7a0716a)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:26:04.489314 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f399a08a-5c20-4992-898d-50f214cb4b5c admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:26:04.544408 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:DELETE=2,keystone:SELECT=96,keystone:INSERT=2 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:26:04.850349 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-f399a08a-5c20-4992-898d-50f214cb4b5c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6efc93f1a5c54ccd9f95b676c7a0716a'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:04.910896 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-f399a08a-5c20-4992-898d-50f214cb4b5c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6efc93f1a5c54ccd9f95b676c7a0716a'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:04.929643 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-f399a08a-5c20-4992-898d-50f214cb4b5c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6efc93f1a5c54ccd9f95b676c7a0716a'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:05.039279 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-f399a08a-5c20-4992-898d-50f214cb4b5c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6efc93f1a5c54ccd9f95b676c7a0716a'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:05.040086 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-f399a08a-5c20-4992-898d-50f214cb4b5c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6efc93f1a5c54ccd9f95b676c7a0716a'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:05.051627 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1234/2443] 10.222.0.55 () {58 vars in 1224 bytes} [Fri Sep 23 15:26:04 2022] DELETE /identity/v3/users/6efc93f1a5c54ccd9f95b676c7a0716a => generated 0 bytes in 747 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:26:05.056104 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9418bdef-bc41-4d1c-b6bb-c7f3e22ee86e None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:05.056923 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9418bdef-bc41-4d1c-b6bb-c7f3e22ee86e None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:05.057361 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9418bdef-bc41-4d1c-b6bb-c7f3e22ee86e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:05.092277 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-9418bdef-bc41-4d1c-b6bb-c7f3e22ee86e None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:26:05.094063 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-9418bdef-bc41-4d1c-b6bb-c7f3e22ee86e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:26:05.178251 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1210/2444] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:26:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:26:05.190354 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f399a08a-5c20-4992-898d-50f214cb4b5c admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:26:05.192026 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-f399a08a-5c20-4992-898d-50f214cb4b5c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:26:05.265647 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f399a08a-5c20-4992-898d-50f214cb4b5c admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:26:05.272970 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c5adc3-1d77-49ee-81ca-c0e5290847ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd259178570864ffeb936ee10dfec6b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:26:05.273969 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c5adc3-1d77-49ee-81ca-c0e5290847ad service neutron] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:05.274090 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c5adc3-1d77-49ee-81ca-c0e5290847ad service neutron] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:05.274283 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c5adc3-1d77-49ee-81ca-c0e5290847ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:05.284693 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c5adc3-1d77-49ee-81ca-c0e5290847ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:26:05.286272 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c5adc3-1d77-49ee-81ca-c0e5290847ad service neutron] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:26:05.313537 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1235/2445] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:26:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:26:05.847501 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9418bdef-bc41-4d1c-b6bb-c7f3e22ee86e None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:26:05.849224 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-9418bdef-bc41-4d1c-b6bb-c7f3e22ee86e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:26:05.863520 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9418bdef-bc41-4d1c-b6bb-c7f3e22ee86e None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:26:05.868390 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3590465a-2be6-49f2-a1ff-7c00f96b485e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:26:05.869119 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3590465a-2be6-49f2-a1ff-7c00f96b485e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:05.869402 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3590465a-2be6-49f2-a1ff-7c00f96b485e admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:05.869684 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3590465a-2be6-49f2-a1ff-7c00f96b485e admin admin] PATH_INFO: `/v3/projects/0d1843be4df843d6a5ae7118346a1b09` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:05.870834 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3590465a-2be6-49f2-a1ff-7c00f96b485e admin admin] RBAC: Authorizing `identity:delete_project(project_id=0d1843be4df843d6a5ae7118346a1b09)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:26:05.872811 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3590465a-2be6-49f2-a1ff-7c00f96b485e admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:26:06.031321 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-3590465a-2be6-49f2-a1ff-7c00f96b485e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0d1843be4df843d6a5ae7118346a1b09', 'request_id': 'req-3590465a-2be6-49f2-a1ff-7c00f96b485e'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:06.038253 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-3590465a-2be6-49f2-a1ff-7c00f96b485e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0d1843be4df843d6a5ae7118346a1b09', 'request_id': 'req-3590465a-2be6-49f2-a1ff-7c00f96b485e'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:06.190135 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-3590465a-2be6-49f2-a1ff-7c00f96b485e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0d1843be4df843d6a5ae7118346a1b09', 'request_id': 'req-3590465a-2be6-49f2-a1ff-7c00f96b485e'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:06.202655 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-3590465a-2be6-49f2-a1ff-7c00f96b485e admin admin] The token cache is being invalidate because project 0d1843be4df843d6a5ae7118346a1b09 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81993) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 15:26:06.202952 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-3590465a-2be6-49f2-a1ff-7c00f96b485e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:06.203634 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-3590465a-2be6-49f2-a1ff-7c00f96b485e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:06.205139 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1211/2446] 10.222.0.55 () {58 vars in 1233 bytes} [Fri Sep 23 15:26:05 2022] DELETE /identity/v3/projects/0d1843be4df843d6a5ae7118346a1b09 => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:26:07.800430 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c5adc3-1d77-49ee-81ca-c0e5290847ad service neutron] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:26:07.803612 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-a2c5adc3-1d77-49ee-81ca-c0e5290847ad service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:26:07.821662 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c5adc3-1d77-49ee-81ca-c0e5290847ad service neutron] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:26:07.919676 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66847aa-f1be-45f2-a377-f51b3b8ab2b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:26:07.920490 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66847aa-f1be-45f2-a377-f51b3b8ab2b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:07.920803 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66847aa-f1be-45f2-a377-f51b3b8ab2b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:07.921115 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66847aa-f1be-45f2-a377-f51b3b8ab2b8 admin admin] PATH_INFO: `/v3/users/83cddbd5031f45499dbf8f12a44e63ed` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:07.922385 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66847aa-f1be-45f2-a377-f51b3b8ab2b8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=83cddbd5031f45499dbf8f12a44e63ed)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:26:07.924350 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66847aa-f1be-45f2-a377-f51b3b8ab2b8 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:26:08.216921 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:DELETE=14,keystone:SELECT=85,keystone:INSERT=1 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:26:08.223085 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-b66847aa-f1be-45f2-a377-f51b3b8ab2b8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '83cddbd5031f45499dbf8f12a44e63ed'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:08.237748 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-b66847aa-f1be-45f2-a377-f51b3b8ab2b8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '83cddbd5031f45499dbf8f12a44e63ed'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:08.255188 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-b66847aa-f1be-45f2-a377-f51b3b8ab2b8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '83cddbd5031f45499dbf8f12a44e63ed'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:08.370836 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-b66847aa-f1be-45f2-a377-f51b3b8ab2b8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '83cddbd5031f45499dbf8f12a44e63ed'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:08.371495 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-b66847aa-f1be-45f2-a377-f51b3b8ab2b8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '83cddbd5031f45499dbf8f12a44e63ed'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:08.378759 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1236/2447] 10.222.0.55 () {58 vars in 1224 bytes} [Fri Sep 23 15:26:07 2022] DELETE /identity/v3/users/83cddbd5031f45499dbf8f12a44e63ed => generated 0 bytes in 579 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:26:08.383611 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e082b26f-c8ca-48a6-b557-f31a5e637370 None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:08.384382 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e082b26f-c8ca-48a6-b557-f31a5e637370 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:08.384616 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e082b26f-c8ca-48a6-b557-f31a5e637370 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:08.410144 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-e082b26f-c8ca-48a6-b557-f31a5e637370 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:26:08.411411 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-e082b26f-c8ca-48a6-b557-f31a5e637370 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:26:08.471368 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:26:08.492978 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1212/2448] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:26:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:26:08.504153 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66847aa-f1be-45f2-a377-f51b3b8ab2b8 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:26:08.505759 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-b66847aa-f1be-45f2-a377-f51b3b8ab2b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:26:08.575907 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66847aa-f1be-45f2-a377-f51b3b8ab2b8 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:26:08.580627 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0070400-b931-417e-b5d1-34a7f47c95f4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd259178570864ffeb936ee10dfec6b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:26:08.581440 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0070400-b931-417e-b5d1-34a7f47c95f4 service neutron] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:08.581595 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0070400-b931-417e-b5d1-34a7f47c95f4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:08.581808 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0070400-b931-417e-b5d1-34a7f47c95f4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:08.592376 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0070400-b931-417e-b5d1-34a7f47c95f4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:26:08.593834 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0070400-b931-417e-b5d1-34a7f47c95f4 service neutron] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:26:08.604310 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1237/2449] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:26:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:26:09.265741 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e082b26f-c8ca-48a6-b557-f31a5e637370 None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:26:09.267332 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-e082b26f-c8ca-48a6-b557-f31a5e637370 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:26:09.281674 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e082b26f-c8ca-48a6-b557-f31a5e637370 None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:26:09.286915 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2045c9f9-c70c-49fb-ba3d-f547f0d1fb48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:26:09.293096 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2045c9f9-c70c-49fb-ba3d-f547f0d1fb48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:09.293096 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2045c9f9-c70c-49fb-ba3d-f547f0d1fb48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:09.293096 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2045c9f9-c70c-49fb-ba3d-f547f0d1fb48 admin admin] PATH_INFO: `/v3/projects/ccf1290dbdf04673949ac453cf3c8939` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:09.293096 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2045c9f9-c70c-49fb-ba3d-f547f0d1fb48 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ccf1290dbdf04673949ac453cf3c8939)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:26:09.293096 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2045c9f9-c70c-49fb-ba3d-f547f0d1fb48 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:26:09.412585 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-2045c9f9-c70c-49fb-ba3d-f547f0d1fb48 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ccf1290dbdf04673949ac453cf3c8939', 'request_id': 'req-2045c9f9-c70c-49fb-ba3d-f547f0d1fb48'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:09.418899 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-2045c9f9-c70c-49fb-ba3d-f547f0d1fb48 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ccf1290dbdf04673949ac453cf3c8939', 'request_id': 'req-2045c9f9-c70c-49fb-ba3d-f547f0d1fb48'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:09.570818 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-2045c9f9-c70c-49fb-ba3d-f547f0d1fb48 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ccf1290dbdf04673949ac453cf3c8939', 'request_id': 'req-2045c9f9-c70c-49fb-ba3d-f547f0d1fb48'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:09.584480 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-2045c9f9-c70c-49fb-ba3d-f547f0d1fb48 admin admin] The token cache is being invalidate because project ccf1290dbdf04673949ac453cf3c8939 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81993) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 15:26:09.584776 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-2045c9f9-c70c-49fb-ba3d-f547f0d1fb48 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:09.585467 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-2045c9f9-c70c-49fb-ba3d-f547f0d1fb48 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:09.587110 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1213/2450] 10.222.0.55 () {58 vars in 1233 bytes} [Fri Sep 23 15:26:09 2022] DELETE /identity/v3/projects/ccf1290dbdf04673949ac453cf3c8939 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:26:09.592043 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea4af64-ba85-4ee6-ab66-556486e5692c None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:09.593133 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea4af64-ba85-4ee6-ab66-556486e5692c None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:09.593133 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea4af64-ba85-4ee6-ab66-556486e5692c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:09.615459 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-dea4af64-ba85-4ee6-ab66-556486e5692c None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:26:09.616718 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-dea4af64-ba85-4ee6-ab66-556486e5692c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:26:09.677862 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1238/2451] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:26:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 15:26:09.685672 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2056e4-02a1-4847-a01a-781500228216 None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:09.685958 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2056e4-02a1-4847-a01a-781500228216 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:09.686762 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2056e4-02a1-4847-a01a-781500228216 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:09.706893 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-9b2056e4-02a1-4847-a01a-781500228216 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:26:09.709031 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-9b2056e4-02a1-4847-a01a-781500228216 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:26:09.715859 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1214/2452] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:26:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:26:09.720782 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea4af64-ba85-4ee6-ab66-556486e5692c None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:26:09.731111 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea4af64-ba85-4ee6-ab66-556486e5692c None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:26:09.736190 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f7b38f-186e-4e9b-a0a4-139e5de7c98a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:26:09.736793 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f7b38f-186e-4e9b-a0a4-139e5de7c98a admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:09.737071 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f7b38f-186e-4e9b-a0a4-139e5de7c98a admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:09.737341 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f7b38f-186e-4e9b-a0a4-139e5de7c98a admin admin] PATH_INFO: `/v3/domains` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:09.738005 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f7b38f-186e-4e9b-a0a4-139e5de7c98a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:26:09.738328 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f7b38f-186e-4e9b-a0a4-139e5de7c98a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:26:09.739997 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f7b38f-186e-4e9b-a0a4-139e5de7c98a admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:26:09.760000 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1239/2453] 10.222.0.55 () {58 vars in 1153 bytes} [Fri Sep 23 15:26:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:26:09.765177 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2056e4-02a1-4847-a01a-781500228216 None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:26:09.775019 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2056e4-02a1-4847-a01a-781500228216 None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:26:09.780618 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651bf5e4-5a5c-4dd5-93b3-80b01765b38f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:26:09.781275 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651bf5e4-5a5c-4dd5-93b3-80b01765b38f admin admin] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:09.787404 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651bf5e4-5a5c-4dd5-93b3-80b01765b38f admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:09.787404 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651bf5e4-5a5c-4dd5-93b3-80b01765b38f admin admin] PATH_INFO: `/v3/projects` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:09.787404 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651bf5e4-5a5c-4dd5-93b3-80b01765b38f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:26:09.787404 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651bf5e4-5a5c-4dd5-93b3-80b01765b38f admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:26:09.913394 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1215/2454] 10.222.0.55 () {60 vars in 1153 bytes} [Fri Sep 23 15:26:09 2022] POST /identity/v3/projects => generated 363 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:26:09.914281 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651bf5e4-5a5c-4dd5-93b3-80b01765b38f admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:26:09.914951 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58deb658-a4a4-4c3e-8572-24b934c8c4fb None None] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:09.915149 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58deb658-a4a4-4c3e-8572-24b934c8c4fb None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:09.915460 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58deb658-a4a4-4c3e-8572-24b934c8c4fb None None] PATH_INFO: `/` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:09.917337 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1240/2455] 10.222.0.55 () {54 vars in 921 bytes} [Fri Sep 23 15:26:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:26:09.928840 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b852303-6744-427b-8267-5b2317b210b7 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:09.928840 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b852303-6744-427b-8267-5b2317b210b7 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:09.928840 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b852303-6744-427b-8267-5b2317b210b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:09.978957 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-5b852303-6744-427b-8267-5b2317b210b7 None None] MFA Rules not processed for user `d259178570864ffeb936ee10dfec6b28`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:26:09.980342 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-5b852303-6744-427b-8267-5b2317b210b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:26:10.002064 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651bf5e4-5a5c-4dd5-93b3-80b01765b38f admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:26:10.007159 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5ea0bf-8d7e-48f3-88cf-345e9ad45220 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:26:10.007779 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5ea0bf-8d7e-48f3-88cf-345e9ad45220 admin admin] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:10.008132 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5ea0bf-8d7e-48f3-88cf-345e9ad45220 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:10.008420 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5ea0bf-8d7e-48f3-88cf-345e9ad45220 admin admin] PATH_INFO: `/v3/users` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:10.009738 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5ea0bf-8d7e-48f3-88cf-345e9ad45220 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:26:10.011736 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5ea0bf-8d7e-48f3-88cf-345e9ad45220 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:26:10.033151 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:26:10.042887 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1241/2456] 10.222.0.55 () {58 vars in 1018 bytes} [Fri Sep 23 15:26:09 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 15:26:10.055562 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b852303-6744-427b-8267-5b2317b210b7 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:26:10.056995 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-5b852303-6744-427b-8267-5b2317b210b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:26:10.068769 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b852303-6744-427b-8267-5b2317b210b7 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:26:10.072556 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71597d3a-1119-41bb-a935-e5ab46d2cbc6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd259178570864ffeb936ee10dfec6b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:26:10.073109 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71597d3a-1119-41bb-a935-e5ab46d2cbc6 service neutron] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:10.073384 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71597d3a-1119-41bb-a935-e5ab46d2cbc6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:10.073659 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71597d3a-1119-41bb-a935-e5ab46d2cbc6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:10.081707 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71597d3a-1119-41bb-a935-e5ab46d2cbc6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:26:10.083171 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71597d3a-1119-41bb-a935-e5ab46d2cbc6 service neutron] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:26:10.091365 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1242/2457] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:26:10 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 23 15:26:10.148673 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1216/2458] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Sep 23 15:26:09 2022] POST /identity/v3/users => generated 282 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:26:10.149921 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71597d3a-1119-41bb-a935-e5ab46d2cbc6 service neutron] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:26:10.161472 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71597d3a-1119-41bb-a935-e5ab46d2cbc6 service neutron] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:26:10.172194 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c89481-cf09-4ceb-8c3a-5cb6f4b5b9eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:26:10.172194 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c89481-cf09-4ceb-8c3a-5cb6f4b5b9eb admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:10.172194 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c89481-cf09-4ceb-8c3a-5cb6f4b5b9eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:10.172194 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c89481-cf09-4ceb-8c3a-5cb6f4b5b9eb admin admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:10.172590 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c89481-cf09-4ceb-8c3a-5cb6f4b5b9eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:26:10.172590 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c89481-cf09-4ceb-8c3a-5cb6f4b5b9eb admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:26:10.178861 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea155c5-fcd6-42e0-a37e-9cc90012d420 None None] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:10.179157 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea155c5-fcd6-42e0-a37e-9cc90012d420 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:10.179539 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea155c5-fcd6-42e0-a37e-9cc90012d420 None None] PATH_INFO: `/` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:10.181011 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1217/2459] 10.222.0.55 () {54 vars in 921 bytes} [Fri Sep 23 15:26:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:26:10.183360 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1243/2460] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Sep 23 15:26:10 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:26:10.188468 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea155c5-fcd6-42e0-a37e-9cc90012d420 None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:26:10.190468 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4af64e-6dff-4a21-a2da-514e7e500cd7 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:10.190764 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4af64e-6dff-4a21-a2da-514e7e500cd7 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:10.191043 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4af64e-6dff-4a21-a2da-514e7e500cd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:10.202694 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea155c5-fcd6-42e0-a37e-9cc90012d420 None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:26:10.208632 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a644c6-df09-4117-8099-68dc8359c512 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:26:10.209338 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a644c6-df09-4117-8099-68dc8359c512 admin admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:10.209759 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a644c6-df09-4117-8099-68dc8359c512 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:10.210053 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a644c6-df09-4117-8099-68dc8359c512 admin admin] PATH_INFO: `/v3/roles` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:10.210698 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a644c6-df09-4117-8099-68dc8359c512 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:26:10.212435 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a644c6-df09-4117-8099-68dc8359c512 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:26:10.213987 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-5a4af64e-6dff-4a21-a2da-514e7e500cd7 None None] MFA Rules not processed for user `b3c6c7a1325a47be924268e1aa1a7a18`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:26:10.215368 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-5a4af64e-6dff-4a21-a2da-514e7e500cd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:26:10.228024 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1218/2461] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Sep 23 15:26:10 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:26:10.230675 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a644c6-df09-4117-8099-68dc8359c512 admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:26:10.242636 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a644c6-df09-4117-8099-68dc8359c512 admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:26:10.248654 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13ea984-e14a-4e49-b8c3-49df1637cb65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:26:10.250052 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13ea984-e14a-4e49-b8c3-49df1637cb65 admin admin] REQUEST_METHOD: `PUT` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:10.250703 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13ea984-e14a-4e49-b8c3-49df1637cb65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:10.250981 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13ea984-e14a-4e49-b8c3-49df1637cb65 admin admin] PATH_INFO: `/v3/projects/4aab9fe86ec041e7b187311bc33714fd/users/94e07ac4a1cf4bbb988a33481cf2c038/roles/af257be2dee54d6b942f58650196582c` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:10.270626 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13ea984-e14a-4e49-b8c3-49df1637cb65 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4aab9fe86ec041e7b187311bc33714fd, user_id=94e07ac4a1cf4bbb988a33481cf2c038, role_id=af257be2dee54d6b942f58650196582c)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:26:10.277326 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13ea984-e14a-4e49-b8c3-49df1637cb65 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:26:10.283088 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:26:10.302378 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1244/2462] 10.222.0.55 () {58 vars in 1018 bytes} [Fri Sep 23 15:26:10 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:26:10.313918 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4af64e-6dff-4a21-a2da-514e7e500cd7 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:26:10.315906 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-5a4af64e-6dff-4a21-a2da-514e7e500cd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:26:10.348095 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4af64e-6dff-4a21-a2da-514e7e500cd7 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:26:10.348095 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c285439f-d2fa-461e-a3e5-3ac2e2611882 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd259178570864ffeb936ee10dfec6b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:26:10.348095 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c285439f-d2fa-461e-a3e5-3ac2e2611882 service neutron] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:10.348095 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c285439f-d2fa-461e-a3e5-3ac2e2611882 service neutron] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:10.348480 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c285439f-d2fa-461e-a3e5-3ac2e2611882 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:10.350918 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c285439f-d2fa-461e-a3e5-3ac2e2611882 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:26:10.352284 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c285439f-d2fa-461e-a3e5-3ac2e2611882 service neutron] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:26:10.366001 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1245/2463] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:26:10 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:26:10.445743 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1219/2464] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Sep 23 15:26:10 2022] PUT /identity/v3/projects/4aab9fe86ec041e7b187311bc33714fd/users/94e07ac4a1cf4bbb988a33481cf2c038/roles/af257be2dee54d6b942f58650196582c => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:26:10.456809 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d44dad-8749-4f6d-96fa-bd846ef31a28 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:10.457091 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d44dad-8749-4f6d-96fa-bd846ef31a28 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:10.467993 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d44dad-8749-4f6d-96fa-bd846ef31a28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:10.490148 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-b3d44dad-8749-4f6d-96fa-bd846ef31a28 None None] MFA Rules not processed for user `94e07ac4a1cf4bbb988a33481cf2c038`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:26:10.491389 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-b3d44dad-8749-4f6d-96fa-bd846ef31a28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:26:10.561422 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1246/2465] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:26:10 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:26:10.566036 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13ea984-e14a-4e49-b8c3-49df1637cb65 admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:26:10.628747 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13ea984-e14a-4e49-b8c3-49df1637cb65 admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:26:10.633244 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59846b3a-7354-4f69-8377-06568d18659a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:26:10.633846 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59846b3a-7354-4f69-8377-06568d18659a admin admin] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:10.634123 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59846b3a-7354-4f69-8377-06568d18659a admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:10.634488 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59846b3a-7354-4f69-8377-06568d18659a admin admin] PATH_INFO: `/v3/projects` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:10.635058 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59846b3a-7354-4f69-8377-06568d18659a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:26:10.637042 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59846b3a-7354-4f69-8377-06568d18659a admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:26:10.774950 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1220/2466] 10.222.0.55 () {60 vars in 1153 bytes} [Fri Sep 23 15:26:10 2022] POST /identity/v3/projects => generated 365 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:26:10.778760 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d44dad-8749-4f6d-96fa-bd846ef31a28 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:26:10.789960 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d44dad-8749-4f6d-96fa-bd846ef31a28 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:26:10.832375 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:26:10.852949 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c664bb-6272-4ae3-8870-74222c8d5e60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:26:10.853640 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c664bb-6272-4ae3-8870-74222c8d5e60 admin admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:10.853925 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c664bb-6272-4ae3-8870-74222c8d5e60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:10.854200 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c664bb-6272-4ae3-8870-74222c8d5e60 admin admin] PATH_INFO: `/v3/users` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:10.854913 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c664bb-6272-4ae3-8870-74222c8d5e60 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:26:10.857022 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c664bb-6272-4ae3-8870-74222c8d5e60 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:26:10.965293 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1247/2467] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Sep 23 15:26:10 2022] POST /identity/v3/users => generated 289 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:26:10.969561 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59846b3a-7354-4f69-8377-06568d18659a admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:26:10.980634 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59846b3a-7354-4f69-8377-06568d18659a admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:26:10.986310 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915f9672-1721-4992-adf5-5e18e23247fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:26:10.986917 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915f9672-1721-4992-adf5-5e18e23247fa admin admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:10.987219 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915f9672-1721-4992-adf5-5e18e23247fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:10.987455 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915f9672-1721-4992-adf5-5e18e23247fa admin admin] PATH_INFO: `/v3/roles` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:10.988127 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915f9672-1721-4992-adf5-5e18e23247fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:26:10.989720 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915f9672-1721-4992-adf5-5e18e23247fa admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:26:11.001591 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1221/2468] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Sep 23 15:26:10 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:26:11.006110 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c664bb-6272-4ae3-8870-74222c8d5e60 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:26:11.016619 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c664bb-6272-4ae3-8870-74222c8d5e60 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:26:11.021535 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd57c38-a785-49be-a1bb-a47c295f8ede admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:26:11.022166 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd57c38-a785-49be-a1bb-a47c295f8ede admin admin] REQUEST_METHOD: `PUT` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:11.022448 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd57c38-a785-49be-a1bb-a47c295f8ede admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:11.022724 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd57c38-a785-49be-a1bb-a47c295f8ede admin admin] PATH_INFO: `/v3/projects/733ac09509434663abf79fdb3d052a62/users/dfc9ee6bb93e4f418865b422022b0fa6/roles/8e2919e79df040e8926f21607818b58c` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:11.039655 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd57c38-a785-49be-a1bb-a47c295f8ede admin admin] RBAC: Authorizing `identity:create_grant(project_id=733ac09509434663abf79fdb3d052a62, user_id=dfc9ee6bb93e4f418865b422022b0fa6, role_id=8e2919e79df040e8926f21607818b58c)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:26:11.043229 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd57c38-a785-49be-a1bb-a47c295f8ede admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:26:11.202583 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1248/2469] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Sep 23 15:26:11 2022] PUT /identity/v3/projects/733ac09509434663abf79fdb3d052a62/users/dfc9ee6bb93e4f418865b422022b0fa6/roles/8e2919e79df040e8926f21607818b58c => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:26:11.203789 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915f9672-1721-4992-adf5-5e18e23247fa admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:26:11.214166 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915f9672-1721-4992-adf5-5e18e23247fa admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:26:11.271374 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674ffe40-407d-477c-9886-62b838f87323 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:26:11.271999 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674ffe40-407d-477c-9886-62b838f87323 admin admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:11.272307 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674ffe40-407d-477c-9886-62b838f87323 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:11.272577 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674ffe40-407d-477c-9886-62b838f87323 admin admin] PATH_INFO: `/v3/roles` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:11.273217 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674ffe40-407d-477c-9886-62b838f87323 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:26:11.274806 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674ffe40-407d-477c-9886-62b838f87323 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:26:11.285544 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1222/2470] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Sep 23 15:26:11 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:26:11.294314 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd57c38-a785-49be-a1bb-a47c295f8ede admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:26:11.300803 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd57c38-a785-49be-a1bb-a47c295f8ede admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:26:11.305468 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5b98fc-0543-4688-9a49-979e586812cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:26:11.306117 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5b98fc-0543-4688-9a49-979e586812cc admin admin] REQUEST_METHOD: `PUT` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:11.306395 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5b98fc-0543-4688-9a49-979e586812cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:11.306666 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5b98fc-0543-4688-9a49-979e586812cc admin admin] PATH_INFO: `/v3/projects/733ac09509434663abf79fdb3d052a62/users/dfc9ee6bb93e4f418865b422022b0fa6/roles/8e2919e79df040e8926f21607818b58c` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:11.308617 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5b98fc-0543-4688-9a49-979e586812cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=733ac09509434663abf79fdb3d052a62, user_id=dfc9ee6bb93e4f418865b422022b0fa6, role_id=8e2919e79df040e8926f21607818b58c)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:26:11.311254 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5b98fc-0543-4688-9a49-979e586812cc admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:26:11.321675 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1249/2471] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Sep 23 15:26:11 2022] PUT /identity/v3/projects/733ac09509434663abf79fdb3d052a62/users/dfc9ee6bb93e4f418865b422022b0fa6/roles/8e2919e79df040e8926f21607818b58c => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:26:11.333859 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c35536a-6be3-46cf-9c60-924de3a0bdc0 None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:11.334151 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c35536a-6be3-46cf-9c60-924de3a0bdc0 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:11.338028 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c35536a-6be3-46cf-9c60-924de3a0bdc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:11.365051 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-3c35536a-6be3-46cf-9c60-924de3a0bdc0 None None] MFA Rules not processed for user `dfc9ee6bb93e4f418865b422022b0fa6`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:26:11.366567 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-3c35536a-6be3-46cf-9c60-924de3a0bdc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:26:11.429741 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:26:11.452170 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1223/2472] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:26:11 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:26:11.463490 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5b98fc-0543-4688-9a49-979e586812cc admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:26:11.465397 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-bc5b98fc-0543-4688-9a49-979e586812cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:26:11.531598 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5b98fc-0543-4688-9a49-979e586812cc admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:26:11.536539 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6816f4-747e-4be8-96ae-f2975fbfe42e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feca6278be904ef8b0dc580a67e477f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:26:11.537179 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6816f4-747e-4be8-96ae-f2975fbfe42e service cinder] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:11.537489 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6816f4-747e-4be8-96ae-f2975fbfe42e service cinder] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:11.537762 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6816f4-747e-4be8-96ae-f2975fbfe42e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:11.549429 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6816f4-747e-4be8-96ae-f2975fbfe42e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:26:11.550924 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6816f4-747e-4be8-96ae-f2975fbfe42e service cinder] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:26:11.572433 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1250/2473] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:26:11 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:26:11.862979 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c35536a-6be3-46cf-9c60-924de3a0bdc0 None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:26:11.872577 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c35536a-6be3-46cf-9c60-924de3a0bdc0 None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:26:11.877269 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b8e3cc-6b97-47a4-a0fd-4096b4c45d91 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feca6278be904ef8b0dc580a67e477f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:26:11.877881 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b8e3cc-6b97-47a4-a0fd-4096b4c45d91 service cinder] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:11.878210 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b8e3cc-6b97-47a4-a0fd-4096b4c45d91 service cinder] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:11.885813 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b8e3cc-6b97-47a4-a0fd-4096b4c45d91 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:11.940396 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b8e3cc-6b97-47a4-a0fd-4096b4c45d91 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:26:11.941830 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b8e3cc-6b97-47a4-a0fd-4096b4c45d91 service cinder] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:26:11.951246 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1224/2474] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:26:11 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 23 15:26:15.873592 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6816f4-747e-4be8-96ae-f2975fbfe42e service cinder] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:26:15.943893 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6816f4-747e-4be8-96ae-f2975fbfe42e service cinder] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:26:15.948320 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82807463-e1ed-4112-895a-eea7ef277105 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:26:15.948944 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82807463-e1ed-4112-895a-eea7ef277105 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:15.949223 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82807463-e1ed-4112-895a-eea7ef277105 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:15.949493 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82807463-e1ed-4112-895a-eea7ef277105 admin admin] PATH_INFO: `/v3/users/94e07ac4a1cf4bbb988a33481cf2c038` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:15.950599 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82807463-e1ed-4112-895a-eea7ef277105 admin admin] RBAC: Authorizing `identity:delete_user(user_id=94e07ac4a1cf4bbb988a33481cf2c038)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:26:15.953058 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82807463-e1ed-4112-895a-eea7ef277105 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:26:16.208913 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:26:16.219621 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-82807463-e1ed-4112-895a-eea7ef277105 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '94e07ac4a1cf4bbb988a33481cf2c038'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:16.231966 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-82807463-e1ed-4112-895a-eea7ef277105 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '94e07ac4a1cf4bbb988a33481cf2c038'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:16.239607 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-82807463-e1ed-4112-895a-eea7ef277105 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '94e07ac4a1cf4bbb988a33481cf2c038'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:16.399009 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-82807463-e1ed-4112-895a-eea7ef277105 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '94e07ac4a1cf4bbb988a33481cf2c038'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:16.399643 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-82807463-e1ed-4112-895a-eea7ef277105 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '94e07ac4a1cf4bbb988a33481cf2c038'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:16.406150 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1251/2475] 10.222.0.55 () {58 vars in 1224 bytes} [Fri Sep 23 15:26:15 2022] DELETE /identity/v3/users/94e07ac4a1cf4bbb988a33481cf2c038 => generated 0 bytes in 533 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:26:16.410968 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793d1b5e-89bd-4132-9207-6535e256dbba None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:16.411462 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793d1b5e-89bd-4132-9207-6535e256dbba None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:16.411592 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793d1b5e-89bd-4132-9207-6535e256dbba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:16.430608 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-793d1b5e-89bd-4132-9207-6535e256dbba None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:26:16.431904 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-793d1b5e-89bd-4132-9207-6535e256dbba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:26:16.493561 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1225/2476] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:26:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:26:16.505281 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82807463-e1ed-4112-895a-eea7ef277105 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:26:16.506842 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-82807463-e1ed-4112-895a-eea7ef277105 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:26:16.566394 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82807463-e1ed-4112-895a-eea7ef277105 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:26:16.570061 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9f0c44-8390-41d1-a71e-ce35d5a4c4be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd259178570864ffeb936ee10dfec6b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:26:16.570675 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9f0c44-8390-41d1-a71e-ce35d5a4c4be service neutron] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:16.570956 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9f0c44-8390-41d1-a71e-ce35d5a4c4be service neutron] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:16.571231 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9f0c44-8390-41d1-a71e-ce35d5a4c4be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:16.581491 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9f0c44-8390-41d1-a71e-ce35d5a4c4be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:26:16.582890 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9f0c44-8390-41d1-a71e-ce35d5a4c4be service neutron] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:26:16.593015 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1252/2477] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:26:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:26:16.759103 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4ca40e-130e-4d10-8b25-05517798a494 None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:16.759387 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4ca40e-130e-4d10-8b25-05517798a494 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:16.759658 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4ca40e-130e-4d10-8b25-05517798a494 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:16.781782 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-ee4ca40e-130e-4d10-8b25-05517798a494 None None] MFA Rules not processed for user `68206a6b4f244d45bf7bf1504396e430`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:26:16.782967 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-ee4ca40e-130e-4d10-8b25-05517798a494 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:26:16.838581 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1226/2478] 10.222.0.55 () {58 vars in 967 bytes} [Fri Sep 23 15:26:16 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:26:16.883604 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9f0c44-8390-41d1-a71e-ce35d5a4c4be service neutron] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:26:16.885557 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-6b9f0c44-8390-41d1-a71e-ce35d5a4c4be service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:26:16.966060 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9f0c44-8390-41d1-a71e-ce35d5a4c4be service neutron] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:26:16.971965 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62f995c-f3d5-4aa3-886a-5dfb21d8b677 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2228c0b49a0249bbaf524a8c4e9e108b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:26:16.972546 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62f995c-f3d5-4aa3-886a-5dfb21d8b677 service swift] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:16.972831 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62f995c-f3d5-4aa3-886a-5dfb21d8b677 service swift] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:16.973110 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62f995c-f3d5-4aa3-886a-5dfb21d8b677 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:16.982288 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62f995c-f3d5-4aa3-886a-5dfb21d8b677 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:26:16.984047 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62f995c-f3d5-4aa3-886a-5dfb21d8b677 service swift] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:26:16.993265 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1253/2479] 10.222.0.55 () {60 vars in 1384 bytes} [Fri Sep 23 15:26:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 23 15:26:17.303724 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4ca40e-130e-4d10-8b25-05517798a494 None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:26:17.305281 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-ee4ca40e-130e-4d10-8b25-05517798a494 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:26:17.322829 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4ca40e-130e-4d10-8b25-05517798a494 None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:26:17.329885 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97faebc3-ab4b-4aa9-818b-eb49b6cf9770 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:26:17.340845 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97faebc3-ab4b-4aa9-818b-eb49b6cf9770 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:17.340845 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97faebc3-ab4b-4aa9-818b-eb49b6cf9770 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:17.340845 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97faebc3-ab4b-4aa9-818b-eb49b6cf9770 admin admin] PATH_INFO: `/v3/projects/4aab9fe86ec041e7b187311bc33714fd` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:17.340845 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97faebc3-ab4b-4aa9-818b-eb49b6cf9770 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4aab9fe86ec041e7b187311bc33714fd)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:26:17.340845 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97faebc3-ab4b-4aa9-818b-eb49b6cf9770 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:26:17.678531 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:26:17.694479 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-97faebc3-ab4b-4aa9-818b-eb49b6cf9770 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4aab9fe86ec041e7b187311bc33714fd', 'request_id': 'req-97faebc3-ab4b-4aa9-818b-eb49b6cf9770'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:17.700206 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-97faebc3-ab4b-4aa9-818b-eb49b6cf9770 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4aab9fe86ec041e7b187311bc33714fd', 'request_id': 'req-97faebc3-ab4b-4aa9-818b-eb49b6cf9770'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:17.830274 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-97faebc3-ab4b-4aa9-818b-eb49b6cf9770 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4aab9fe86ec041e7b187311bc33714fd', 'request_id': 'req-97faebc3-ab4b-4aa9-818b-eb49b6cf9770'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:17.847271 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-97faebc3-ab4b-4aa9-818b-eb49b6cf9770 admin admin] The token cache is being invalidate because project 4aab9fe86ec041e7b187311bc33714fd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81993) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 15:26:17.847271 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-97faebc3-ab4b-4aa9-818b-eb49b6cf9770 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:17.847271 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-97faebc3-ab4b-4aa9-818b-eb49b6cf9770 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:17.847271 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1227/2480] 10.222.0.55 () {58 vars in 1233 bytes} [Fri Sep 23 15:26:17 2022] DELETE /identity/v3/projects/4aab9fe86ec041e7b187311bc33714fd => generated 0 bytes in 543 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:26:17.850086 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62f995c-f3d5-4aa3-886a-5dfb21d8b677 service swift] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:26:17.861585 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62f995c-f3d5-4aa3-886a-5dfb21d8b677 service swift] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:26:17.909720 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:26:17.924356 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048f06cc-2536-4862-b5f5-2ffb71ad63c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:26:17.925019 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048f06cc-2536-4862-b5f5-2ffb71ad63c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:17.925348 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048f06cc-2536-4862-b5f5-2ffb71ad63c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:17.925622 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048f06cc-2536-4862-b5f5-2ffb71ad63c7 admin admin] PATH_INFO: `/v3/users/dfc9ee6bb93e4f418865b422022b0fa6` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:17.926780 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048f06cc-2536-4862-b5f5-2ffb71ad63c7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dfc9ee6bb93e4f418865b422022b0fa6)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:26:17.928951 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048f06cc-2536-4862-b5f5-2ffb71ad63c7 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:26:18.222004 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-048f06cc-2536-4862-b5f5-2ffb71ad63c7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dfc9ee6bb93e4f418865b422022b0fa6'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:18.238334 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-048f06cc-2536-4862-b5f5-2ffb71ad63c7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dfc9ee6bb93e4f418865b422022b0fa6'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:18.246724 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-048f06cc-2536-4862-b5f5-2ffb71ad63c7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dfc9ee6bb93e4f418865b422022b0fa6'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:18.351076 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-048f06cc-2536-4862-b5f5-2ffb71ad63c7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dfc9ee6bb93e4f418865b422022b0fa6'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:18.351725 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-048f06cc-2536-4862-b5f5-2ffb71ad63c7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dfc9ee6bb93e4f418865b422022b0fa6'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:18.358637 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1254/2481] 10.222.0.55 () {58 vars in 1224 bytes} [Fri Sep 23 15:26:17 2022] DELETE /identity/v3/users/dfc9ee6bb93e4f418865b422022b0fa6 => generated 0 bytes in 509 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:26:18.958041 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97faebc3-ab4b-4aa9-818b-eb49b6cf9770 admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:26:18.964646 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-97faebc3-ab4b-4aa9-818b-eb49b6cf9770 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:26:18.983091 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97faebc3-ab4b-4aa9-818b-eb49b6cf9770 admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:26:19.047659 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abbf11e-247b-42a9-b14f-b20249df6e8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:26:19.048350 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abbf11e-247b-42a9-b14f-b20249df6e8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:19.048630 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abbf11e-247b-42a9-b14f-b20249df6e8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:19.048903 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abbf11e-247b-42a9-b14f-b20249df6e8c admin admin] PATH_INFO: `/v3/projects/733ac09509434663abf79fdb3d052a62` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:19.050328 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abbf11e-247b-42a9-b14f-b20249df6e8c admin admin] RBAC: Authorizing `identity:delete_project(project_id=733ac09509434663abf79fdb3d052a62)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:26:19.058542 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abbf11e-247b-42a9-b14f-b20249df6e8c admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:26:19.261941 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-0abbf11e-247b-42a9-b14f-b20249df6e8c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '733ac09509434663abf79fdb3d052a62', 'request_id': 'req-0abbf11e-247b-42a9-b14f-b20249df6e8c'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:19.268287 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-0abbf11e-247b-42a9-b14f-b20249df6e8c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '733ac09509434663abf79fdb3d052a62', 'request_id': 'req-0abbf11e-247b-42a9-b14f-b20249df6e8c'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:19.403058 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-0abbf11e-247b-42a9-b14f-b20249df6e8c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '733ac09509434663abf79fdb3d052a62', 'request_id': 'req-0abbf11e-247b-42a9-b14f-b20249df6e8c'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:19.416432 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-0abbf11e-247b-42a9-b14f-b20249df6e8c admin admin] The token cache is being invalidate because project 733ac09509434663abf79fdb3d052a62 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81993) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 15:26:19.416815 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-0abbf11e-247b-42a9-b14f-b20249df6e8c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:19.417526 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-0abbf11e-247b-42a9-b14f-b20249df6e8c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:19.419082 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1228/2482] 10.222.0.55 () {58 vars in 1233 bytes} [Fri Sep 23 15:26:18 2022] DELETE /identity/v3/projects/733ac09509434663abf79fdb3d052a62 => generated 0 bytes in 462 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:26:19.425940 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62538b51-eb7b-4edc-9207-68036f408564 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:19.426433 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62538b51-eb7b-4edc-9207-68036f408564 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:19.426554 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62538b51-eb7b-4edc-9207-68036f408564 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:19.447905 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-62538b51-eb7b-4edc-9207-68036f408564 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:26:19.449170 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-62538b51-eb7b-4edc-9207-68036f408564 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:26:19.515795 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1255/2483] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:26:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:26:19.523748 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245455d4-bd23-4906-8c79-61aeef216c87 None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:19.524152 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245455d4-bd23-4906-8c79-61aeef216c87 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:19.524495 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245455d4-bd23-4906-8c79-61aeef216c87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:19.545281 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-245455d4-bd23-4906-8c79-61aeef216c87 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:26:19.546789 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-245455d4-bd23-4906-8c79-61aeef216c87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:26:19.555081 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1229/2484] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:26:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:26:19.559438 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62538b51-eb7b-4edc-9207-68036f408564 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:26:19.570090 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62538b51-eb7b-4edc-9207-68036f408564 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:26:19.574944 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973f8c04-862e-4874-9021-a59412c4516f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:26:19.575555 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973f8c04-862e-4874-9021-a59412c4516f admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:19.575836 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973f8c04-862e-4874-9021-a59412c4516f admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:19.576149 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973f8c04-862e-4874-9021-a59412c4516f admin admin] PATH_INFO: `/v3/domains` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:19.577578 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973f8c04-862e-4874-9021-a59412c4516f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:26:19.577843 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973f8c04-862e-4874-9021-a59412c4516f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:26:19.579526 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973f8c04-862e-4874-9021-a59412c4516f admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:26:19.595620 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1256/2485] 10.222.0.55 () {58 vars in 1153 bytes} [Fri Sep 23 15:26:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:26:19.597133 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245455d4-bd23-4906-8c79-61aeef216c87 None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:26:19.606760 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245455d4-bd23-4906-8c79-61aeef216c87 None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:26:19.618945 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67d4020-da5d-4069-a649-24bd890f3343 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:26:19.619895 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67d4020-da5d-4069-a649-24bd890f3343 admin admin] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:19.620288 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67d4020-da5d-4069-a649-24bd890f3343 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:19.620615 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67d4020-da5d-4069-a649-24bd890f3343 admin admin] PATH_INFO: `/v3/projects` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:19.621279 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67d4020-da5d-4069-a649-24bd890f3343 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:26:19.623954 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67d4020-da5d-4069-a649-24bd890f3343 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:26:19.726711 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1230/2486] 10.222.0.55 () {60 vars in 1153 bytes} [Fri Sep 23 15:26:19 2022] POST /identity/v3/projects => generated 379 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:26:19.731022 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973f8c04-862e-4874-9021-a59412c4516f admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:26:19.742734 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973f8c04-862e-4874-9021-a59412c4516f admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:26:19.792422 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:26:19.801699 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f27385-b5ea-43c0-a9e0-59aaf37f2127 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:26:19.802305 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f27385-b5ea-43c0-a9e0-59aaf37f2127 admin admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:19.808990 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f27385-b5ea-43c0-a9e0-59aaf37f2127 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:19.808990 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f27385-b5ea-43c0-a9e0-59aaf37f2127 admin admin] PATH_INFO: `/v3/users` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:19.808990 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f27385-b5ea-43c0-a9e0-59aaf37f2127 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:26:19.808990 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f27385-b5ea-43c0-a9e0-59aaf37f2127 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:26:19.945395 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1257/2487] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Sep 23 15:26:19 2022] POST /identity/v3/users => generated 290 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:26:19.952190 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67d4020-da5d-4069-a649-24bd890f3343 admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:26:19.961468 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67d4020-da5d-4069-a649-24bd890f3343 admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:26:19.966524 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b84bf3-3514-41aa-8461-ca0d30b6860c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:26:19.966835 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b84bf3-3514-41aa-8461-ca0d30b6860c admin admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:19.967122 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b84bf3-3514-41aa-8461-ca0d30b6860c admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:19.967569 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b84bf3-3514-41aa-8461-ca0d30b6860c admin admin] PATH_INFO: `/v3/roles` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:19.968346 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b84bf3-3514-41aa-8461-ca0d30b6860c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:26:19.969945 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b84bf3-3514-41aa-8461-ca0d30b6860c admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:26:19.981524 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1231/2488] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Sep 23 15:26:19 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:26:19.990503 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f27385-b5ea-43c0-a9e0-59aaf37f2127 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:26:20.002334 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f27385-b5ea-43c0-a9e0-59aaf37f2127 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:26:20.008201 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1d522f-ea7f-4aa9-8c22-d7b272e9ae95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:26:20.008830 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1d522f-ea7f-4aa9-8c22-d7b272e9ae95 admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:20.009116 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1d522f-ea7f-4aa9-8c22-d7b272e9ae95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:20.009394 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1d522f-ea7f-4aa9-8c22-d7b272e9ae95 admin admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:20.010082 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1d522f-ea7f-4aa9-8c22-d7b272e9ae95 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:26:20.012033 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1d522f-ea7f-4aa9-8c22-d7b272e9ae95 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:26:20.036539 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b84bf3-3514-41aa-8461-ca0d30b6860c admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:26:20.036789 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1258/2489] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Sep 23 15:26:19 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:26:20.047799 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b84bf3-3514-41aa-8461-ca0d30b6860c admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:26:20.052724 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b129f14-2251-478c-bcaa-0d142bb14b70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:26:20.053548 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b129f14-2251-478c-bcaa-0d142bb14b70 admin admin] REQUEST_METHOD: `PUT` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:20.053830 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b129f14-2251-478c-bcaa-0d142bb14b70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:20.054106 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b129f14-2251-478c-bcaa-0d142bb14b70 admin admin] PATH_INFO: `/v3/projects/00a0ad83a1d04ab483ac66328ba4189e/users/4f2d3dfe206a42509e1087ace605d929/roles/af257be2dee54d6b942f58650196582c` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:20.057693 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:26:20.076179 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b129f14-2251-478c-bcaa-0d142bb14b70 admin admin] RBAC: Authorizing `identity:create_grant(project_id=00a0ad83a1d04ab483ac66328ba4189e, user_id=4f2d3dfe206a42509e1087ace605d929, role_id=af257be2dee54d6b942f58650196582c)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:26:20.079892 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b129f14-2251-478c-bcaa-0d142bb14b70 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:26:20.291360 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1232/2490] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Sep 23 15:26:20 2022] PUT /identity/v3/projects/00a0ad83a1d04ab483ac66328ba4189e/users/4f2d3dfe206a42509e1087ace605d929/roles/af257be2dee54d6b942f58650196582c => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:26:20.303012 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb0c307-722d-4d4d-acda-0346a88ab22e None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:20.303236 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb0c307-722d-4d4d-acda-0346a88ab22e None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:20.303513 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb0c307-722d-4d4d-acda-0346a88ab22e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:20.333316 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-7bb0c307-722d-4d4d-acda-0346a88ab22e None None] MFA Rules not processed for user `4f2d3dfe206a42509e1087ace605d929`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:26:20.334593 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-7bb0c307-722d-4d4d-acda-0346a88ab22e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:26:20.408053 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1259/2491] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:26:20 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:26:20.412577 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b129f14-2251-478c-bcaa-0d142bb14b70 admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:26:20.476780 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b129f14-2251-478c-bcaa-0d142bb14b70 admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:26:20.481600 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf510a24-4036-4fa1-9de3-714df8fe1309 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:26:20.482316 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf510a24-4036-4fa1-9de3-714df8fe1309 admin admin] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:20.483200 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf510a24-4036-4fa1-9de3-714df8fe1309 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:20.483200 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf510a24-4036-4fa1-9de3-714df8fe1309 admin admin] PATH_INFO: `/v3/projects` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:20.484002 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf510a24-4036-4fa1-9de3-714df8fe1309 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:26:20.486131 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf510a24-4036-4fa1-9de3-714df8fe1309 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:26:20.656779 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1233/2492] 10.222.0.55 () {60 vars in 1153 bytes} [Fri Sep 23 15:26:20 2022] POST /identity/v3/projects => generated 381 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:26:20.660422 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb0c307-722d-4d4d-acda-0346a88ab22e None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:26:20.728202 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb0c307-722d-4d4d-acda-0346a88ab22e None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:26:20.803056 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19af73ed-2689-4ffe-9733-cc464410cf24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:26:20.803789 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19af73ed-2689-4ffe-9733-cc464410cf24 admin admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:20.804181 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19af73ed-2689-4ffe-9733-cc464410cf24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:20.804881 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19af73ed-2689-4ffe-9733-cc464410cf24 admin admin] PATH_INFO: `/v3/users` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:20.805246 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19af73ed-2689-4ffe-9733-cc464410cf24 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:26:20.807318 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19af73ed-2689-4ffe-9733-cc464410cf24 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:26:20.954670 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1260/2493] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Sep 23 15:26:20 2022] POST /identity/v3/users => generated 297 bytes in 295 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:26:20.958784 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf510a24-4036-4fa1-9de3-714df8fe1309 admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:26:21.013054 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf510a24-4036-4fa1-9de3-714df8fe1309 admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:26:21.019309 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a1ed72-a609-4def-a69d-29aea6418fde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:26:21.019931 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a1ed72-a609-4def-a69d-29aea6418fde admin admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:21.020236 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a1ed72-a609-4def-a69d-29aea6418fde admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:21.020512 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a1ed72-a609-4def-a69d-29aea6418fde admin admin] PATH_INFO: `/v3/roles` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:21.021154 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a1ed72-a609-4def-a69d-29aea6418fde admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:26:21.022933 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a1ed72-a609-4def-a69d-29aea6418fde admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:26:21.034872 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1234/2494] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Sep 23 15:26:20 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:26:21.039113 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19af73ed-2689-4ffe-9733-cc464410cf24 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:26:21.072467 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19af73ed-2689-4ffe-9733-cc464410cf24 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:26:21.089224 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f02ec1a-b036-4cbf-b5ea-8ef518e05e4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:26:21.093076 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f02ec1a-b036-4cbf-b5ea-8ef518e05e4b admin admin] REQUEST_METHOD: `PUT` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:21.093076 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f02ec1a-b036-4cbf-b5ea-8ef518e05e4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:21.093076 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f02ec1a-b036-4cbf-b5ea-8ef518e05e4b admin admin] PATH_INFO: `/v3/projects/a0a8ec1235e74c46b91619b69476d5de/users/57018b5360c84c7d90b99fe37d169f68/roles/8e2919e79df040e8926f21607818b58c` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:21.106172 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:26:21.115611 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f02ec1a-b036-4cbf-b5ea-8ef518e05e4b admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0a8ec1235e74c46b91619b69476d5de, user_id=57018b5360c84c7d90b99fe37d169f68, role_id=8e2919e79df040e8926f21607818b58c)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:26:21.118934 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f02ec1a-b036-4cbf-b5ea-8ef518e05e4b admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:26:21.243064 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1261/2495] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Sep 23 15:26:21 2022] PUT /identity/v3/projects/a0a8ec1235e74c46b91619b69476d5de/users/57018b5360c84c7d90b99fe37d169f68/roles/8e2919e79df040e8926f21607818b58c => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:26:21.245031 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a1ed72-a609-4def-a69d-29aea6418fde admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:26:21.259212 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a1ed72-a609-4def-a69d-29aea6418fde admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:26:21.346097 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0331cfe-9615-4a24-a3cb-a5338c669c52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:26:21.346915 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0331cfe-9615-4a24-a3cb-a5338c669c52 admin admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:21.347297 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0331cfe-9615-4a24-a3cb-a5338c669c52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:21.347720 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0331cfe-9615-4a24-a3cb-a5338c669c52 admin admin] PATH_INFO: `/v3/roles` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:21.348521 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0331cfe-9615-4a24-a3cb-a5338c669c52 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:26:21.350429 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0331cfe-9615-4a24-a3cb-a5338c669c52 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:26:21.356735 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:26:21.364385 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1235/2496] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Sep 23 15:26:21 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:26:21.376632 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f02ec1a-b036-4cbf-b5ea-8ef518e05e4b admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:26:21.387673 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f02ec1a-b036-4cbf-b5ea-8ef518e05e4b admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:26:21.393598 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804219ac-b571-41e4-80fc-d5bc921dc0b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:26:21.394725 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804219ac-b571-41e4-80fc-d5bc921dc0b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:21.395173 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804219ac-b571-41e4-80fc-d5bc921dc0b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:21.395786 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804219ac-b571-41e4-80fc-d5bc921dc0b2 admin admin] PATH_INFO: `/v3/projects/a0a8ec1235e74c46b91619b69476d5de/users/57018b5360c84c7d90b99fe37d169f68/roles/8e2919e79df040e8926f21607818b58c` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:21.397983 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804219ac-b571-41e4-80fc-d5bc921dc0b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0a8ec1235e74c46b91619b69476d5de, user_id=57018b5360c84c7d90b99fe37d169f68, role_id=8e2919e79df040e8926f21607818b58c)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:26:21.400870 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804219ac-b571-41e4-80fc-d5bc921dc0b2 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:26:21.412750 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1262/2497] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Sep 23 15:26:21 2022] PUT /identity/v3/projects/a0a8ec1235e74c46b91619b69476d5de/users/57018b5360c84c7d90b99fe37d169f68/roles/8e2919e79df040e8926f21607818b58c => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 15:26:21.424887 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2f7016-b3a3-437c-b397-3250f61cf102 None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:21.425188 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2f7016-b3a3-437c-b397-3250f61cf102 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:21.425446 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2f7016-b3a3-437c-b397-3250f61cf102 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:21.463493 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-8c2f7016-b3a3-437c-b397-3250f61cf102 None None] MFA Rules not processed for user `57018b5360c84c7d90b99fe37d169f68`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:26:21.466333 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-8c2f7016-b3a3-437c-b397-3250f61cf102 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:26:21.542988 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1236/2498] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:26:21 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:26:21.560223 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804219ac-b571-41e4-80fc-d5bc921dc0b2 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:26:21.562640 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-804219ac-b571-41e4-80fc-d5bc921dc0b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:26:21.625341 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804219ac-b571-41e4-80fc-d5bc921dc0b2 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:26:21.629127 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4995b055-01df-4f79-847c-03683d9fd672 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feca6278be904ef8b0dc580a67e477f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:26:21.633242 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4995b055-01df-4f79-847c-03683d9fd672 service cinder] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:21.633242 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4995b055-01df-4f79-847c-03683d9fd672 service cinder] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:21.633242 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4995b055-01df-4f79-847c-03683d9fd672 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:21.642629 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4995b055-01df-4f79-847c-03683d9fd672 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:26:21.644218 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4995b055-01df-4f79-847c-03683d9fd672 service cinder] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:26:21.654317 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1263/2499] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:26:21 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:26:22.868833 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2f7016-b3a3-437c-b397-3250f61cf102 None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:26:22.885286 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2f7016-b3a3-437c-b397-3250f61cf102 None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:26:22.897623 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e706dc4-2eed-4b7c-af1b-8871a5308f77 tempest-VolumeQuotaClassesTest-1412683804 tempest-VolumeQuotaClassesTest-1412683804-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57018b5360c84c7d90b99fe37d169f68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0a8ec1235e74c46b91619b69476d5de', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:26:22.898266 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e706dc4-2eed-4b7c-af1b-8871a5308f77 tempest-VolumeQuotaClassesTest-1412683804 tempest-VolumeQuotaClassesTest-1412683804-project-admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:22.898547 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e706dc4-2eed-4b7c-af1b-8871a5308f77 tempest-VolumeQuotaClassesTest-1412683804 tempest-VolumeQuotaClassesTest-1412683804-project-admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:22.898891 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e706dc4-2eed-4b7c-af1b-8871a5308f77 tempest-VolumeQuotaClassesTest-1412683804 tempest-VolumeQuotaClassesTest-1412683804-project-admin] PATH_INFO: `/v3/domains` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:22.899560 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e706dc4-2eed-4b7c-af1b-8871a5308f77 tempest-VolumeQuotaClassesTest-1412683804 tempest-VolumeQuotaClassesTest-1412683804-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:26:22.899864 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e706dc4-2eed-4b7c-af1b-8871a5308f77 tempest-VolumeQuotaClassesTest-1412683804 tempest-VolumeQuotaClassesTest-1412683804-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:26:22.901504 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e706dc4-2eed-4b7c-af1b-8871a5308f77 tempest-VolumeQuotaClassesTest-1412683804 tempest-VolumeQuotaClassesTest-1412683804-project-admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:26:22.916804 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1237/2500] 10.222.0.55 () {58 vars in 1153 bytes} [Fri Sep 23 15:26:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:26:22.923985 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4995b055-01df-4f79-847c-03683d9fd672 service cinder] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:26:22.933320 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4995b055-01df-4f79-847c-03683d9fd672 service cinder] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:26:22.938811 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac9bd55-3f93-45b0-b7c4-79097461221c tempest-VolumeQuotaClassesTest-1412683804 tempest-VolumeQuotaClassesTest-1412683804-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57018b5360c84c7d90b99fe37d169f68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0a8ec1235e74c46b91619b69476d5de', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:26:22.939412 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac9bd55-3f93-45b0-b7c4-79097461221c tempest-VolumeQuotaClassesTest-1412683804 tempest-VolumeQuotaClassesTest-1412683804-project-admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:22.939695 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac9bd55-3f93-45b0-b7c4-79097461221c tempest-VolumeQuotaClassesTest-1412683804 tempest-VolumeQuotaClassesTest-1412683804-project-admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:22.940134 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac9bd55-3f93-45b0-b7c4-79097461221c tempest-VolumeQuotaClassesTest-1412683804 tempest-VolumeQuotaClassesTest-1412683804-project-admin] PATH_INFO: `/v3/projects` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:22.940773 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac9bd55-3f93-45b0-b7c4-79097461221c tempest-VolumeQuotaClassesTest-1412683804 tempest-VolumeQuotaClassesTest-1412683804-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:26:22.942712 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac9bd55-3f93-45b0-b7c4-79097461221c tempest-VolumeQuotaClassesTest-1412683804 tempest-VolumeQuotaClassesTest-1412683804-project-admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:26:23.093651 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1264/2501] 10.222.0.55 () {60 vars in 1153 bytes} [Fri Sep 23 15:26:22 2022] POST /identity/v3/projects => generated 355 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:26:23.098609 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e706dc4-2eed-4b7c-af1b-8871a5308f77 tempest-VolumeQuotaClassesTest-1412683804 tempest-VolumeQuotaClassesTest-1412683804-project-admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:26:23.109959 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e706dc4-2eed-4b7c-af1b-8871a5308f77 tempest-VolumeQuotaClassesTest-1412683804 tempest-VolumeQuotaClassesTest-1412683804-project-admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:26:23.191577 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23cdec4-4ef9-45f1-90ca-502df7033bf7 tempest-VolumeQuotaClassesTest-1412683804 tempest-VolumeQuotaClassesTest-1412683804-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57018b5360c84c7d90b99fe37d169f68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0a8ec1235e74c46b91619b69476d5de', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:26:23.192628 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23cdec4-4ef9-45f1-90ca-502df7033bf7 tempest-VolumeQuotaClassesTest-1412683804 tempest-VolumeQuotaClassesTest-1412683804-project-admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:23.192906 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23cdec4-4ef9-45f1-90ca-502df7033bf7 tempest-VolumeQuotaClassesTest-1412683804 tempest-VolumeQuotaClassesTest-1412683804-project-admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:23.193176 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23cdec4-4ef9-45f1-90ca-502df7033bf7 tempest-VolumeQuotaClassesTest-1412683804 tempest-VolumeQuotaClassesTest-1412683804-project-admin] PATH_INFO: `/v3/domains` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:23.193905 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23cdec4-4ef9-45f1-90ca-502df7033bf7 tempest-VolumeQuotaClassesTest-1412683804 tempest-VolumeQuotaClassesTest-1412683804-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:26:23.194207 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23cdec4-4ef9-45f1-90ca-502df7033bf7 tempest-VolumeQuotaClassesTest-1412683804 tempest-VolumeQuotaClassesTest-1412683804-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:26:23.196128 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23cdec4-4ef9-45f1-90ca-502df7033bf7 tempest-VolumeQuotaClassesTest-1412683804 tempest-VolumeQuotaClassesTest-1412683804-project-admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:26:23.213693 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1238/2502] 10.222.0.55 () {58 vars in 1153 bytes} [Fri Sep 23 15:26:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 15:26:23.252434 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac9bd55-3f93-45b0-b7c4-79097461221c tempest-VolumeQuotaClassesTest-1412683804 tempest-VolumeQuotaClassesTest-1412683804-project-admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:26:23.265880 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac9bd55-3f93-45b0-b7c4-79097461221c tempest-VolumeQuotaClassesTest-1412683804 tempest-VolumeQuotaClassesTest-1412683804-project-admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:26:23.272657 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e06222-a6f0-4f6c-92cf-4c97f8033622 tempest-VolumeQuotaClassesTest-1412683804 tempest-VolumeQuotaClassesTest-1412683804-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57018b5360c84c7d90b99fe37d169f68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0a8ec1235e74c46b91619b69476d5de', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:26:23.273304 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e06222-a6f0-4f6c-92cf-4c97f8033622 tempest-VolumeQuotaClassesTest-1412683804 tempest-VolumeQuotaClassesTest-1412683804-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:23.273641 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e06222-a6f0-4f6c-92cf-4c97f8033622 tempest-VolumeQuotaClassesTest-1412683804 tempest-VolumeQuotaClassesTest-1412683804-project-admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:23.273963 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e06222-a6f0-4f6c-92cf-4c97f8033622 tempest-VolumeQuotaClassesTest-1412683804 tempest-VolumeQuotaClassesTest-1412683804-project-admin] PATH_INFO: `/v3/projects/38679f1e4ba242ada0de0429a06b5309` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:23.276867 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e06222-a6f0-4f6c-92cf-4c97f8033622 tempest-VolumeQuotaClassesTest-1412683804 tempest-VolumeQuotaClassesTest-1412683804-project-admin] RBAC: Authorizing `identity:delete_project(project_id=38679f1e4ba242ada0de0429a06b5309)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:26:23.279252 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e06222-a6f0-4f6c-92cf-4c97f8033622 tempest-VolumeQuotaClassesTest-1412683804 tempest-VolumeQuotaClassesTest-1412683804-project-admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:26:23.477720 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-03e06222-a6f0-4f6c-92cf-4c97f8033622 tempest-VolumeQuotaClassesTest-1412683804 tempest-VolumeQuotaClassesTest-1412683804-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '38679f1e4ba242ada0de0429a06b5309', 'request_id': 'req-03e06222-a6f0-4f6c-92cf-4c97f8033622'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:23.484544 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-03e06222-a6f0-4f6c-92cf-4c97f8033622 tempest-VolumeQuotaClassesTest-1412683804 tempest-VolumeQuotaClassesTest-1412683804-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '38679f1e4ba242ada0de0429a06b5309', 'request_id': 'req-03e06222-a6f0-4f6c-92cf-4c97f8033622'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:23.490696 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-03e06222-a6f0-4f6c-92cf-4c97f8033622 tempest-VolumeQuotaClassesTest-1412683804 tempest-VolumeQuotaClassesTest-1412683804-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '38679f1e4ba242ada0de0429a06b5309', 'request_id': 'req-03e06222-a6f0-4f6c-92cf-4c97f8033622'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:23.589425 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-03e06222-a6f0-4f6c-92cf-4c97f8033622 tempest-VolumeQuotaClassesTest-1412683804 tempest-VolumeQuotaClassesTest-1412683804-project-admin] The token cache is being invalidate because project 38679f1e4ba242ada0de0429a06b5309 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81994) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 15:26:23.589718 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-03e06222-a6f0-4f6c-92cf-4c97f8033622 tempest-VolumeQuotaClassesTest-1412683804 tempest-VolumeQuotaClassesTest-1412683804-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:23.590381 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-03e06222-a6f0-4f6c-92cf-4c97f8033622 tempest-VolumeQuotaClassesTest-1412683804 tempest-VolumeQuotaClassesTest-1412683804-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:23.592137 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1265/2503] 10.222.0.55 () {58 vars in 1233 bytes} [Fri Sep 23 15:26:23 2022] DELETE /identity/v3/projects/38679f1e4ba242ada0de0429a06b5309 => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:26:24.509020 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23cdec4-4ef9-45f1-90ca-502df7033bf7 tempest-VolumeQuotaClassesTest-1412683804 tempest-VolumeQuotaClassesTest-1412683804-project-admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:26:24.523656 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-c23cdec4-4ef9-45f1-90ca-502df7033bf7 tempest-VolumeQuotaClassesTest-1412683804 tempest-VolumeQuotaClassesTest-1412683804-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:26:24.633520 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23cdec4-4ef9-45f1-90ca-502df7033bf7 tempest-VolumeQuotaClassesTest-1412683804 tempest-VolumeQuotaClassesTest-1412683804-project-admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:26:24.645119 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:26:24.725615 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a9242f-b96c-4e99-a48e-ba4e69d0c245 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:26:24.726344 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a9242f-b96c-4e99-a48e-ba4e69d0c245 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:24.726699 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a9242f-b96c-4e99-a48e-ba4e69d0c245 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:24.727052 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a9242f-b96c-4e99-a48e-ba4e69d0c245 admin admin] PATH_INFO: `/v3/users/4f2d3dfe206a42509e1087ace605d929` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:24.728454 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a9242f-b96c-4e99-a48e-ba4e69d0c245 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4f2d3dfe206a42509e1087ace605d929)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:26:24.730579 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a9242f-b96c-4e99-a48e-ba4e69d0c245 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:26:24.940651 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-43a9242f-b96c-4e99-a48e-ba4e69d0c245 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4f2d3dfe206a42509e1087ace605d929'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:24.955096 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-43a9242f-b96c-4e99-a48e-ba4e69d0c245 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4f2d3dfe206a42509e1087ace605d929'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:24.962314 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-43a9242f-b96c-4e99-a48e-ba4e69d0c245 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4f2d3dfe206a42509e1087ace605d929'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:25.105293 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-43a9242f-b96c-4e99-a48e-ba4e69d0c245 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4f2d3dfe206a42509e1087ace605d929'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:25.106112 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-43a9242f-b96c-4e99-a48e-ba4e69d0c245 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4f2d3dfe206a42509e1087ace605d929'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:25.120677 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1239/2504] 10.222.0.55 () {58 vars in 1224 bytes} [Fri Sep 23 15:26:24 2022] DELETE /identity/v3/users/4f2d3dfe206a42509e1087ace605d929 => generated 0 bytes in 610 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:26:25.122738 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb366915-0b7a-4e17-aa74-e9c5559e9b1c None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:25.123224 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb366915-0b7a-4e17-aa74-e9c5559e9b1c None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:25.123604 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb366915-0b7a-4e17-aa74-e9c5559e9b1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:25.191938 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-bb366915-0b7a-4e17-aa74-e9c5559e9b1c None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:26:25.193402 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-bb366915-0b7a-4e17-aa74-e9c5559e9b1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:26:25.247257 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:26:25.340318 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1266/2505] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:26:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:26:25.349281 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a9242f-b96c-4e99-a48e-ba4e69d0c245 admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:26:25.351012 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-43a9242f-b96c-4e99-a48e-ba4e69d0c245 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:26:25.482498 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a9242f-b96c-4e99-a48e-ba4e69d0c245 admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:26:25.489747 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c98c22b-d7e0-46a5-8b16-679def52d2d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd259178570864ffeb936ee10dfec6b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:26:25.490581 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c98c22b-d7e0-46a5-8b16-679def52d2d3 service neutron] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:25.490925 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c98c22b-d7e0-46a5-8b16-679def52d2d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:25.492044 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c98c22b-d7e0-46a5-8b16-679def52d2d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:25.506952 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c98c22b-d7e0-46a5-8b16-679def52d2d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:26:25.508760 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c98c22b-d7e0-46a5-8b16-679def52d2d3 service neutron] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:26:25.519108 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:26:25.526701 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1240/2506] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:26:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:26:26.109350 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb366915-0b7a-4e17-aa74-e9c5559e9b1c None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:26:26.110295 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-bb366915-0b7a-4e17-aa74-e9c5559e9b1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:26:26.129431 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb366915-0b7a-4e17-aa74-e9c5559e9b1c None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:26:26.137906 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0472cf-65c9-491e-8c5f-8feb3844d0cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:26:26.139135 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0472cf-65c9-491e-8c5f-8feb3844d0cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:26.140705 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0472cf-65c9-491e-8c5f-8feb3844d0cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:26.141136 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0472cf-65c9-491e-8c5f-8feb3844d0cc admin admin] PATH_INFO: `/v3/projects/00a0ad83a1d04ab483ac66328ba4189e` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:26.142678 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0472cf-65c9-491e-8c5f-8feb3844d0cc admin admin] RBAC: Authorizing `identity:delete_project(project_id=00a0ad83a1d04ab483ac66328ba4189e)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:26:26.144819 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0472cf-65c9-491e-8c5f-8feb3844d0cc admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:26:26.257082 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-fc0472cf-65c9-491e-8c5f-8feb3844d0cc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '00a0ad83a1d04ab483ac66328ba4189e', 'request_id': 'req-fc0472cf-65c9-491e-8c5f-8feb3844d0cc'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:26.264884 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-fc0472cf-65c9-491e-8c5f-8feb3844d0cc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '00a0ad83a1d04ab483ac66328ba4189e', 'request_id': 'req-fc0472cf-65c9-491e-8c5f-8feb3844d0cc'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:26.273583 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-fc0472cf-65c9-491e-8c5f-8feb3844d0cc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '00a0ad83a1d04ab483ac66328ba4189e', 'request_id': 'req-fc0472cf-65c9-491e-8c5f-8feb3844d0cc'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:26.396253 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-fc0472cf-65c9-491e-8c5f-8feb3844d0cc admin admin] The token cache is being invalidate because project 00a0ad83a1d04ab483ac66328ba4189e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81994) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 15:26:26.400988 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-fc0472cf-65c9-491e-8c5f-8feb3844d0cc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:26.400988 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-fc0472cf-65c9-491e-8c5f-8feb3844d0cc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:26.400988 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1267/2507] 10.222.0.55 () {58 vars in 1233 bytes} [Fri Sep 23 15:26:26 2022] DELETE /identity/v3/projects/00a0ad83a1d04ab483ac66328ba4189e => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:26:26.403759 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c98c22b-d7e0-46a5-8b16-679def52d2d3 service neutron] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:26:26.416403 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c98c22b-d7e0-46a5-8b16-679def52d2d3 service neutron] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:26:26.506145 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ead990a-cef1-4a19-bd08-e337a4a91a67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:26:26.508273 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ead990a-cef1-4a19-bd08-e337a4a91a67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:26.508386 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ead990a-cef1-4a19-bd08-e337a4a91a67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:26.508571 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ead990a-cef1-4a19-bd08-e337a4a91a67 admin admin] PATH_INFO: `/v3/users/57018b5360c84c7d90b99fe37d169f68` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:26.509836 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ead990a-cef1-4a19-bd08-e337a4a91a67 admin admin] RBAC: Authorizing `identity:delete_user(user_id=57018b5360c84c7d90b99fe37d169f68)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:26:26.511867 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ead990a-cef1-4a19-bd08-e337a4a91a67 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:26:26.753482 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-9ead990a-cef1-4a19-bd08-e337a4a91a67 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '57018b5360c84c7d90b99fe37d169f68'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:26.766322 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-9ead990a-cef1-4a19-bd08-e337a4a91a67 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '57018b5360c84c7d90b99fe37d169f68'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:26.773417 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-9ead990a-cef1-4a19-bd08-e337a4a91a67 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '57018b5360c84c7d90b99fe37d169f68'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:26.853711 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-9ead990a-cef1-4a19-bd08-e337a4a91a67 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '57018b5360c84c7d90b99fe37d169f68'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:26.854753 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-9ead990a-cef1-4a19-bd08-e337a4a91a67 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '57018b5360c84c7d90b99fe37d169f68'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:26.868678 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1241/2508] 10.222.0.55 () {58 vars in 1224 bytes} [Fri Sep 23 15:26:26 2022] DELETE /identity/v3/users/57018b5360c84c7d90b99fe37d169f68 => generated 0 bytes in 465 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:26:27.814375 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0472cf-65c9-491e-8c5f-8feb3844d0cc admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:26:27.815929 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-fc0472cf-65c9-491e-8c5f-8feb3844d0cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:26:27.831228 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0472cf-65c9-491e-8c5f-8feb3844d0cc admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:26:27.894736 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa50058-e683-4ced-938b-00d4177a8418 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:26:27.895435 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa50058-e683-4ced-938b-00d4177a8418 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:27.895653 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa50058-e683-4ced-938b-00d4177a8418 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:27.895928 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa50058-e683-4ced-938b-00d4177a8418 admin admin] PATH_INFO: `/v3/users/deda513f4c6448368cfa79203cf0d8d8` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:27.900191 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ead990a-cef1-4a19-bd08-e337a4a91a67 admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:26:27.900734 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa50058-e683-4ced-938b-00d4177a8418 admin admin] RBAC: Authorizing `identity:delete_user(user_id=deda513f4c6448368cfa79203cf0d8d8)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:26:27.901727 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-9ead990a-cef1-4a19-bd08-e337a4a91a67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:26:27.902655 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa50058-e683-4ced-938b-00d4177a8418 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:26:27.924568 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ead990a-cef1-4a19-bd08-e337a4a91a67 admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:26:27.932937 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edb98d3-4c8b-4e13-be4a-bbefc67894ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:26:27.933661 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edb98d3-4c8b-4e13-be4a-bbefc67894ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:27.938190 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edb98d3-4c8b-4e13-be4a-bbefc67894ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:27.938190 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edb98d3-4c8b-4e13-be4a-bbefc67894ed admin admin] PATH_INFO: `/v3/projects/a0a8ec1235e74c46b91619b69476d5de` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:27.938190 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edb98d3-4c8b-4e13-be4a-bbefc67894ed admin admin] RBAC: Authorizing `identity:delete_project(project_id=a0a8ec1235e74c46b91619b69476d5de)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:26:27.939315 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edb98d3-4c8b-4e13-be4a-bbefc67894ed admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:26:28.168182 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-9edb98d3-4c8b-4e13-be4a-bbefc67894ed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a0a8ec1235e74c46b91619b69476d5de', 'request_id': 'req-9edb98d3-4c8b-4e13-be4a-bbefc67894ed'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:28.168658 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-3aa50058-e683-4ced-938b-00d4177a8418 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'deda513f4c6448368cfa79203cf0d8d8'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:28.172264 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:26:28.176693 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-9edb98d3-4c8b-4e13-be4a-bbefc67894ed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a0a8ec1235e74c46b91619b69476d5de', 'request_id': 'req-9edb98d3-4c8b-4e13-be4a-bbefc67894ed'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:28.188334 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-3aa50058-e683-4ced-938b-00d4177a8418 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'deda513f4c6448368cfa79203cf0d8d8'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:28.195951 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-3aa50058-e683-4ced-938b-00d4177a8418 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'deda513f4c6448368cfa79203cf0d8d8'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:28.321308 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-9edb98d3-4c8b-4e13-be4a-bbefc67894ed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a0a8ec1235e74c46b91619b69476d5de', 'request_id': 'req-9edb98d3-4c8b-4e13-be4a-bbefc67894ed'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:28.334545 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-9edb98d3-4c8b-4e13-be4a-bbefc67894ed admin admin] The token cache is being invalidate because project a0a8ec1235e74c46b91619b69476d5de was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81993) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 15:26:28.334965 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-9edb98d3-4c8b-4e13-be4a-bbefc67894ed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:28.335634 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-9edb98d3-4c8b-4e13-be4a-bbefc67894ed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:28.337279 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1242/2509] 10.222.0.55 () {58 vars in 1233 bytes} [Fri Sep 23 15:26:27 2022] DELETE /identity/v3/projects/a0a8ec1235e74c46b91619b69476d5de => generated 0 bytes in 438 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 15:26:28.344142 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79327ec-1412-40a0-b979-39106ca89bcf None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:28.353765 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79327ec-1412-40a0-b979-39106ca89bcf None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:28.353765 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79327ec-1412-40a0-b979-39106ca89bcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:28.353765 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:26:28.415341 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-b79327ec-1412-40a0-b979-39106ca89bcf None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:26:28.416612 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-b79327ec-1412-40a0-b979-39106ca89bcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:26:28.424589 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-3aa50058-e683-4ced-938b-00d4177a8418 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'deda513f4c6448368cfa79203cf0d8d8'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:28.425210 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-3aa50058-e683-4ced-938b-00d4177a8418 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'deda513f4c6448368cfa79203cf0d8d8'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:28.431878 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1268/2510] 10.222.0.55 () {58 vars in 1224 bytes} [Fri Sep 23 15:26:27 2022] DELETE /identity/v3/users/deda513f4c6448368cfa79203cf0d8d8 => generated 0 bytes in 618 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:26:28.500961 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1243/2511] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:26:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:26:28.509158 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55a5ef8-3fe8-4ae2-8e70-f5bbcd723623 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:28.509915 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55a5ef8-3fe8-4ae2-8e70-f5bbcd723623 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:28.509915 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55a5ef8-3fe8-4ae2-8e70-f5bbcd723623 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:28.541256 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-c55a5ef8-3fe8-4ae2-8e70-f5bbcd723623 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:26:28.542823 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-c55a5ef8-3fe8-4ae2-8e70-f5bbcd723623 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:26:28.661147 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1269/2512] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:26:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:26:28.665537 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79327ec-1412-40a0-b979-39106ca89bcf None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:26:28.677381 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79327ec-1412-40a0-b979-39106ca89bcf None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:26:28.682406 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8847ce0a-15fd-4022-b097-80dd476de332 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:26:28.683147 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8847ce0a-15fd-4022-b097-80dd476de332 admin admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:28.683505 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8847ce0a-15fd-4022-b097-80dd476de332 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:28.683856 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8847ce0a-15fd-4022-b097-80dd476de332 admin admin] PATH_INFO: `/v3/domains` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:28.684705 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8847ce0a-15fd-4022-b097-80dd476de332 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:26:28.685015 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8847ce0a-15fd-4022-b097-80dd476de332 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:26:28.686862 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8847ce0a-15fd-4022-b097-80dd476de332 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:26:28.704742 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1244/2513] 10.222.0.55 () {58 vars in 1153 bytes} [Fri Sep 23 15:26:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:26:28.706458 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55a5ef8-3fe8-4ae2-8e70-f5bbcd723623 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:26:28.716621 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55a5ef8-3fe8-4ae2-8e70-f5bbcd723623 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:26:28.721507 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f305de05-28bf-4b06-ab99-a7d93ec78d9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:26:28.722160 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f305de05-28bf-4b06-ab99-a7d93ec78d9b admin admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:28.722466 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f305de05-28bf-4b06-ab99-a7d93ec78d9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:28.722772 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f305de05-28bf-4b06-ab99-a7d93ec78d9b admin admin] PATH_INFO: `/v3/projects` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:28.723425 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f305de05-28bf-4b06-ab99-a7d93ec78d9b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:26:28.725436 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f305de05-28bf-4b06-ab99-a7d93ec78d9b admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:26:28.763339 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8847ce0a-15fd-4022-b097-80dd476de332 admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:26:28.765577 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-8847ce0a-15fd-4022-b097-80dd476de332 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:26:28.782627 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8847ce0a-15fd-4022-b097-80dd476de332 admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:26:28.787377 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054e5a85-a5aa-4482-af27-6406391b258f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:26:28.788127 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054e5a85-a5aa-4482-af27-6406391b258f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:28.796138 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054e5a85-a5aa-4482-af27-6406391b258f admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:28.796834 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054e5a85-a5aa-4482-af27-6406391b258f admin admin] PATH_INFO: `/v3/projects/efbab8ef4fbd4c61811d0fc046290937` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:28.799164 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054e5a85-a5aa-4482-af27-6406391b258f admin admin] RBAC: Authorizing `identity:delete_project(project_id=efbab8ef4fbd4c61811d0fc046290937)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:26:28.801933 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054e5a85-a5aa-4482-af27-6406391b258f admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:26:28.844217 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1270/2514] 10.222.0.55 () {60 vars in 1153 bytes} [Fri Sep 23 15:26:28 2022] POST /identity/v3/projects => generated 379 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:26:28.848644 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f305de05-28bf-4b06-ab99-a7d93ec78d9b admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:26:28.929020 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f305de05-28bf-4b06-ab99-a7d93ec78d9b admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:26:28.947098 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-054e5a85-a5aa-4482-af27-6406391b258f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'efbab8ef4fbd4c61811d0fc046290937', 'request_id': 'req-054e5a85-a5aa-4482-af27-6406391b258f'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:28.954703 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-054e5a85-a5aa-4482-af27-6406391b258f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'efbab8ef4fbd4c61811d0fc046290937', 'request_id': 'req-054e5a85-a5aa-4482-af27-6406391b258f'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:28.996827 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af84742-2eea-474d-bcd0-bc5ccdc0c9f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:26:28.997728 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af84742-2eea-474d-bcd0-bc5ccdc0c9f0 admin admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:28.998066 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af84742-2eea-474d-bcd0-bc5ccdc0c9f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:28.998465 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af84742-2eea-474d-bcd0-bc5ccdc0c9f0 admin admin] PATH_INFO: `/v3/users` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:28.999362 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af84742-2eea-474d-bcd0-bc5ccdc0c9f0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:26:29.001316 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af84742-2eea-474d-bcd0-bc5ccdc0c9f0 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:26:29.075394 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-054e5a85-a5aa-4482-af27-6406391b258f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'efbab8ef4fbd4c61811d0fc046290937', 'request_id': 'req-054e5a85-a5aa-4482-af27-6406391b258f'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:29.089651 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-054e5a85-a5aa-4482-af27-6406391b258f admin admin] The token cache is being invalidate because project efbab8ef4fbd4c61811d0fc046290937 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81993) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 15:26:29.089946 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-054e5a85-a5aa-4482-af27-6406391b258f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:29.090625 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-054e5a85-a5aa-4482-af27-6406391b258f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:29.092305 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1245/2515] 10.222.0.55 () {58 vars in 1233 bytes} [Fri Sep 23 15:26:28 2022] DELETE /identity/v3/projects/efbab8ef4fbd4c61811d0fc046290937 => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 15:26:29.096462 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054e5a85-a5aa-4482-af27-6406391b258f admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:26:29.097673 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-054e5a85-a5aa-4482-af27-6406391b258f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:26:29.110487 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054e5a85-a5aa-4482-af27-6406391b258f admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:26:29.115165 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6a46cd-ac75-4666-a1ab-53e25a648799 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:26:29.115776 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6a46cd-ac75-4666-a1ab-53e25a648799 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:29.116092 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6a46cd-ac75-4666-a1ab-53e25a648799 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:29.116367 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6a46cd-ac75-4666-a1ab-53e25a648799 admin admin] PATH_INFO: `/v3/users/ed55d058f3f042709386d5742c3c6b7a` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:29.117530 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6a46cd-ac75-4666-a1ab-53e25a648799 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ed55d058f3f042709386d5742c3c6b7a)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:26:29.119437 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6a46cd-ac75-4666-a1ab-53e25a648799 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:26:29.142892 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1271/2516] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Sep 23 15:26:28 2022] POST /identity/v3/users => generated 290 bytes in 295 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 23 15:26:29.147010 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af84742-2eea-474d-bcd0-bc5ccdc0c9f0 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:26:29.148766 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-2af84742-2eea-474d-bcd0-bc5ccdc0c9f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:26:29.160485 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af84742-2eea-474d-bcd0-bc5ccdc0c9f0 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:26:29.165186 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab71ac04-3a81-491d-b015-6ae8a82667e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:26:29.165792 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab71ac04-3a81-491d-b015-6ae8a82667e7 admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:29.166070 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab71ac04-3a81-491d-b015-6ae8a82667e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:29.166340 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab71ac04-3a81-491d-b015-6ae8a82667e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:29.166986 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab71ac04-3a81-491d-b015-6ae8a82667e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:26:29.168704 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab71ac04-3a81-491d-b015-6ae8a82667e7 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:26:29.179664 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1272/2517] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Sep 23 15:26:29 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 23 15:26:29.183607 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab71ac04-3a81-491d-b015-6ae8a82667e7 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:26:29.193960 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab71ac04-3a81-491d-b015-6ae8a82667e7 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:26:29.198611 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8578f4-e35a-4410-81e6-8681ae141198 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:26:29.199188 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8578f4-e35a-4410-81e6-8681ae141198 admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:29.199522 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8578f4-e35a-4410-81e6-8681ae141198 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:29.199804 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8578f4-e35a-4410-81e6-8681ae141198 admin admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:29.200494 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8578f4-e35a-4410-81e6-8681ae141198 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:26:29.202857 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8578f4-e35a-4410-81e6-8681ae141198 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:26:29.214684 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1273/2518] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Sep 23 15:26:29 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 23 15:26:29.218957 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8578f4-e35a-4410-81e6-8681ae141198 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:26:29.228926 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8578f4-e35a-4410-81e6-8681ae141198 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:26:29.233606 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32ce0e6-b3c5-4fd3-bf14-471ef51062d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:26:29.234236 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32ce0e6-b3c5-4fd3-bf14-471ef51062d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:29.234522 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32ce0e6-b3c5-4fd3-bf14-471ef51062d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:29.234799 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32ce0e6-b3c5-4fd3-bf14-471ef51062d1 admin admin] PATH_INFO: `/v3/projects/fc9693b3782346509e69e05f8a072615/users/32d96158cee44725b1458a9d3b1190bb/roles/af257be2dee54d6b942f58650196582c` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:29.250372 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32ce0e6-b3c5-4fd3-bf14-471ef51062d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc9693b3782346509e69e05f8a072615, user_id=32d96158cee44725b1458a9d3b1190bb, role_id=af257be2dee54d6b942f58650196582c)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:26:29.253363 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32ce0e6-b3c5-4fd3-bf14-471ef51062d1 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:26:29.353337 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1274/2519] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Sep 23 15:26:29 2022] PUT /identity/v3/projects/fc9693b3782346509e69e05f8a072615/users/32d96158cee44725b1458a9d3b1190bb/roles/af257be2dee54d6b942f58650196582c => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 15:26:29.364983 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6f56f3-57b3-4a7a-831f-705717a358fd None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:29.365270 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6f56f3-57b3-4a7a-831f-705717a358fd None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:29.365574 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6f56f3-57b3-4a7a-831f-705717a358fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:29.391115 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-fc6a46cd-ac75-4666-a1ab-53e25a648799 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ed55d058f3f042709386d5742c3c6b7a'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:29.397311 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-3e6f56f3-57b3-4a7a-831f-705717a358fd None None] MFA Rules not processed for user `32d96158cee44725b1458a9d3b1190bb`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:26:29.398795 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-3e6f56f3-57b3-4a7a-831f-705717a358fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:26:29.403995 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-fc6a46cd-ac75-4666-a1ab-53e25a648799 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ed55d058f3f042709386d5742c3c6b7a'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:29.411372 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-fc6a46cd-ac75-4666-a1ab-53e25a648799 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ed55d058f3f042709386d5742c3c6b7a'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:29.464448 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1275/2520] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:26:29 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 15:26:29.469955 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6f56f3-57b3-4a7a-831f-705717a358fd None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:26:29.509544 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-fc6a46cd-ac75-4666-a1ab-53e25a648799 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ed55d058f3f042709386d5742c3c6b7a'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:29.510162 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-fc6a46cd-ac75-4666-a1ab-53e25a648799 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ed55d058f3f042709386d5742c3c6b7a'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:29.518602 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1246/2521] 10.222.0.55 () {58 vars in 1224 bytes} [Fri Sep 23 15:26:29 2022] DELETE /identity/v3/users/ed55d058f3f042709386d5742c3c6b7a => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 15:26:29.531623 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6f56f3-57b3-4a7a-831f-705717a358fd None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:26:29.589132 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59627c5c-2750-40f6-8690-d9c7df7684fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:26:29.589744 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59627c5c-2750-40f6-8690-d9c7df7684fa admin admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:29.590150 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59627c5c-2750-40f6-8690-d9c7df7684fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:29.590296 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59627c5c-2750-40f6-8690-d9c7df7684fa admin admin] PATH_INFO: `/v3/projects` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:29.590952 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59627c5c-2750-40f6-8690-d9c7df7684fa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:26:29.593779 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59627c5c-2750-40f6-8690-d9c7df7684fa admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:26:29.612738 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=95,keystone:DELETE=3,keystone:INSERT=2 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:26:29.670899 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1276/2522] 10.222.0.55 () {60 vars in 1153 bytes} [Fri Sep 23 15:26:29 2022] POST /identity/v3/projects => generated 385 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 23 15:26:29.675344 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6a46cd-ac75-4666-a1ab-53e25a648799 admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:26:29.677053 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-fc6a46cd-ac75-4666-a1ab-53e25a648799 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:26:29.690737 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6a46cd-ac75-4666-a1ab-53e25a648799 admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:26:29.702794 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:26:29.754715 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c7ee48-3f61-4eca-96a6-24e05ae8fe33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:26:29.755333 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c7ee48-3f61-4eca-96a6-24e05ae8fe33 admin admin] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:29.755615 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c7ee48-3f61-4eca-96a6-24e05ae8fe33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:29.755890 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c7ee48-3f61-4eca-96a6-24e05ae8fe33 admin admin] PATH_INFO: `/v3/users` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:29.756555 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c7ee48-3f61-4eca-96a6-24e05ae8fe33 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:26:29.758492 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c7ee48-3f61-4eca-96a6-24e05ae8fe33 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:26:29.770977 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59627c5c-2750-40f6-8690-d9c7df7684fa admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:26:29.773505 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-59627c5c-2750-40f6-8690-d9c7df7684fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:26:29.788358 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59627c5c-2750-40f6-8690-d9c7df7684fa admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:26:29.794154 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64228c07-def5-4f3e-ae75-ad710e8cdc30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:26:29.794635 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64228c07-def5-4f3e-ae75-ad710e8cdc30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:29.795035 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64228c07-def5-4f3e-ae75-ad710e8cdc30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:29.795383 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64228c07-def5-4f3e-ae75-ad710e8cdc30 admin admin] PATH_INFO: `/v3/projects/5faba0985c464030a7b238905671c767` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:29.796771 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64228c07-def5-4f3e-ae75-ad710e8cdc30 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5faba0985c464030a7b238905671c767)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:26:29.798824 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64228c07-def5-4f3e-ae75-ad710e8cdc30 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:26:29.863828 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1247/2523] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Sep 23 15:26:29 2022] POST /identity/v3/users => generated 293 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 23 15:26:29.868286 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c7ee48-3f61-4eca-96a6-24e05ae8fe33 admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:26:29.879232 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c7ee48-3f61-4eca-96a6-24e05ae8fe33 admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:26:29.884147 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfe6dcd-413a-4de9-a2dc-d67bf2f6f9e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:26:29.884755 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfe6dcd-413a-4de9-a2dc-d67bf2f6f9e2 admin admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:29.885029 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfe6dcd-413a-4de9-a2dc-d67bf2f6f9e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:29.885381 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfe6dcd-413a-4de9-a2dc-d67bf2f6f9e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:29.886023 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbfe6dcd-413a-4de9-a2dc-d67bf2f6f9e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:26:29.887607 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbfe6dcd-413a-4de9-a2dc-d67bf2f6f9e2 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:26:29.898942 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1248/2524] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Sep 23 15:26:29 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:26:29.902904 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfe6dcd-413a-4de9-a2dc-d67bf2f6f9e2 admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:26:29.912811 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfe6dcd-413a-4de9-a2dc-d67bf2f6f9e2 admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:26:29.917416 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54061907-aa1c-423d-8795-893e7c3ae3fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:26:29.917993 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54061907-aa1c-423d-8795-893e7c3ae3fb admin admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:29.918270 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54061907-aa1c-423d-8795-893e7c3ae3fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:29.918538 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54061907-aa1c-423d-8795-893e7c3ae3fb admin admin] PATH_INFO: `/v3/roles` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:29.919281 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54061907-aa1c-423d-8795-893e7c3ae3fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:26:29.921138 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54061907-aa1c-423d-8795-893e7c3ae3fb admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:26:29.932519 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1249/2525] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Sep 23 15:26:29 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 23 15:26:29.937119 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54061907-aa1c-423d-8795-893e7c3ae3fb admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:26:29.947734 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54061907-aa1c-423d-8795-893e7c3ae3fb admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:26:29.953364 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfe1a6b-8a87-4ca3-a81a-3541cfd89269 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:26:29.954114 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfe1a6b-8a87-4ca3-a81a-3541cfd89269 admin admin] REQUEST_METHOD: `PUT` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:29.954391 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfe1a6b-8a87-4ca3-a81a-3541cfd89269 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:29.954663 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfe1a6b-8a87-4ca3-a81a-3541cfd89269 admin admin] PATH_INFO: `/v3/projects/442f001261ea48be83436bfc65116e65/users/b15cbfe9a3644c7aacc5ebcc77bd5fdc/roles/af257be2dee54d6b942f58650196582c` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:29.971151 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfe1a6b-8a87-4ca3-a81a-3541cfd89269 admin admin] RBAC: Authorizing `identity:create_grant(project_id=442f001261ea48be83436bfc65116e65, user_id=b15cbfe9a3644c7aacc5ebcc77bd5fdc, role_id=af257be2dee54d6b942f58650196582c)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:26:29.973867 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfe1a6b-8a87-4ca3-a81a-3541cfd89269 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:26:29.979191 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-64228c07-def5-4f3e-ae75-ad710e8cdc30 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5faba0985c464030a7b238905671c767', 'request_id': 'req-64228c07-def5-4f3e-ae75-ad710e8cdc30'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:29.986187 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-64228c07-def5-4f3e-ae75-ad710e8cdc30 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5faba0985c464030a7b238905671c767', 'request_id': 'req-64228c07-def5-4f3e-ae75-ad710e8cdc30'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:29.991918 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-64228c07-def5-4f3e-ae75-ad710e8cdc30 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5faba0985c464030a7b238905671c767', 'request_id': 'req-64228c07-def5-4f3e-ae75-ad710e8cdc30'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:30.086634 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1250/2526] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Sep 23 15:26:29 2022] PUT /identity/v3/projects/442f001261ea48be83436bfc65116e65/users/b15cbfe9a3644c7aacc5ebcc77bd5fdc/roles/af257be2dee54d6b942f58650196582c => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 15:26:30.096897 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f237575c-63f4-4757-8cb1-bb7aa7349044 None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:30.097180 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f237575c-63f4-4757-8cb1-bb7aa7349044 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:30.097450 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f237575c-63f4-4757-8cb1-bb7aa7349044 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:30.128065 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-f237575c-63f4-4757-8cb1-bb7aa7349044 None None] MFA Rules not processed for user `b15cbfe9a3644c7aacc5ebcc77bd5fdc`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:26:30.129626 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-f237575c-63f4-4757-8cb1-bb7aa7349044 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:26:30.154346 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-64228c07-def5-4f3e-ae75-ad710e8cdc30 admin admin] The token cache is being invalidate because project 5faba0985c464030a7b238905671c767 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81994) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 15:26:30.155901 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-64228c07-def5-4f3e-ae75-ad710e8cdc30 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:30.156241 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-64228c07-def5-4f3e-ae75-ad710e8cdc30 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:30.158287 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1277/2527] 10.222.0.55 () {58 vars in 1233 bytes} [Fri Sep 23 15:26:29 2022] DELETE /identity/v3/projects/5faba0985c464030a7b238905671c767 => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:26:30.162907 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2df325-1f79-4c8c-957f-332eb4cca52a None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:30.163066 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2df325-1f79-4c8c-957f-332eb4cca52a None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:30.163630 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2df325-1f79-4c8c-957f-332eb4cca52a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:30.187331 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-ff2df325-1f79-4c8c-957f-332eb4cca52a None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:26:30.192461 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-ff2df325-1f79-4c8c-957f-332eb4cca52a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:26:30.199771 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1251/2528] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:26:30 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 15:26:30.205401 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f237575c-63f4-4757-8cb1-bb7aa7349044 None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:26:30.206866 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-f237575c-63f4-4757-8cb1-bb7aa7349044 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:26:30.249738 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1278/2529] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:26:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 15:26:30.257519 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e29dc4c-f620-4128-baaf-dafb48929e85 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:30.257808 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e29dc4c-f620-4128-baaf-dafb48929e85 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:30.258087 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e29dc4c-f620-4128-baaf-dafb48929e85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:30.275187 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f237575c-63f4-4757-8cb1-bb7aa7349044 None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:26:30.278074 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-8e29dc4c-f620-4128-baaf-dafb48929e85 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:26:30.279259 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-8e29dc4c-f620-4128-baaf-dafb48929e85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:26:30.280246 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f146a4-3144-4cf8-b3bf-86ea8b77d02f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:26:30.280860 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f146a4-3144-4cf8-b3bf-86ea8b77d02f admin admin] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:30.281143 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f146a4-3144-4cf8-b3bf-86ea8b77d02f admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:30.281419 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f146a4-3144-4cf8-b3bf-86ea8b77d02f admin admin] PATH_INFO: `/v3/projects` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:30.282053 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f146a4-3144-4cf8-b3bf-86ea8b77d02f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:26:30.284187 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f146a4-3144-4cf8-b3bf-86ea8b77d02f admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:26:30.288209 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1279/2530] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:26:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 15:26:30.292661 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e29dc4c-f620-4128-baaf-dafb48929e85 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:26:30.302365 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e29dc4c-f620-4128-baaf-dafb48929e85 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:26:30.306972 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4b54ef-4631-44d5-8df2-6097dc6422fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:26:30.312929 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4b54ef-4631-44d5-8df2-6097dc6422fe admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:30.312929 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4b54ef-4631-44d5-8df2-6097dc6422fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:30.312929 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4b54ef-4631-44d5-8df2-6097dc6422fe admin admin] PATH_INFO: `/v3/domains` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:30.312929 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4b54ef-4631-44d5-8df2-6097dc6422fe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:26:30.312929 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4b54ef-4631-44d5-8df2-6097dc6422fe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:26:30.312929 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4b54ef-4631-44d5-8df2-6097dc6422fe admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:26:30.324216 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1280/2531] 10.222.0.55 () {58 vars in 1153 bytes} [Fri Sep 23 15:26:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 15:26:30.328728 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4b54ef-4631-44d5-8df2-6097dc6422fe admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:26:30.338419 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4b54ef-4631-44d5-8df2-6097dc6422fe admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:26:30.343264 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d8c19f-fc0c-4912-bd35-a0ef403ac7b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:26:30.343856 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d8c19f-fc0c-4912-bd35-a0ef403ac7b5 admin admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:30.344200 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d8c19f-fc0c-4912-bd35-a0ef403ac7b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:30.344483 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d8c19f-fc0c-4912-bd35-a0ef403ac7b5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:30.353945 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d8c19f-fc0c-4912-bd35-a0ef403ac7b5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:26:30.353945 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d8c19f-fc0c-4912-bd35-a0ef403ac7b5 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:26:30.395088 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1252/2532] 10.222.0.55 () {60 vars in 1153 bytes} [Fri Sep 23 15:26:30 2022] POST /identity/v3/projects => generated 385 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 23 15:26:30.398696 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f146a4-3144-4cf8-b3bf-86ea8b77d02f admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:26:30.458953 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f146a4-3144-4cf8-b3bf-86ea8b77d02f admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:26:30.463575 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3b7cfa-a925-420f-9983-d77cf754452c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:26:30.464213 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3b7cfa-a925-420f-9983-d77cf754452c admin admin] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:30.464499 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3b7cfa-a925-420f-9983-d77cf754452c admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:30.464765 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3b7cfa-a925-420f-9983-d77cf754452c admin admin] PATH_INFO: `/v3/users` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:30.465389 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3b7cfa-a925-420f-9983-d77cf754452c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:26:30.467564 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3b7cfa-a925-420f-9983-d77cf754452c admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:26:30.531681 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1281/2533] 10.222.0.55 () {60 vars in 1153 bytes} [Fri Sep 23 15:26:30 2022] POST /identity/v3/projects => generated 377 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 23 15:26:30.532668 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d8c19f-fc0c-4912-bd35-a0ef403ac7b5 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:26:30.561953 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:26:30.598391 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d8c19f-fc0c-4912-bd35-a0ef403ac7b5 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:26:30.604859 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184a44b5-f161-4d4a-9fea-4926dc19a457 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:26:30.605483 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184a44b5-f161-4d4a-9fea-4926dc19a457 admin admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:30.605998 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184a44b5-f161-4d4a-9fea-4926dc19a457 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:30.608503 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184a44b5-f161-4d4a-9fea-4926dc19a457 admin admin] PATH_INFO: `/v3/users` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:30.608503 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184a44b5-f161-4d4a-9fea-4926dc19a457 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:26:30.612976 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184a44b5-f161-4d4a-9fea-4926dc19a457 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:26:30.613559 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1253/2534] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Sep 23 15:26:30 2022] POST /identity/v3/users => generated 299 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 23 15:26:30.614715 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3b7cfa-a925-420f-9983-d77cf754452c admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:26:30.627550 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3b7cfa-a925-420f-9983-d77cf754452c admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:26:30.632950 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae4f260-f7ae-40f4-beb8-68a569a38a48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:26:30.633603 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae4f260-f7ae-40f4-beb8-68a569a38a48 admin admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:30.633885 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae4f260-f7ae-40f4-beb8-68a569a38a48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:30.634460 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae4f260-f7ae-40f4-beb8-68a569a38a48 admin admin] PATH_INFO: `/v3/roles` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:30.635348 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae4f260-f7ae-40f4-beb8-68a569a38a48 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:26:30.637113 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:DELETE=4,keystone:SELECT=91,keystone:INSERT=5 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:26:30.642102 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae4f260-f7ae-40f4-beb8-68a569a38a48 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:26:30.650351 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1254/2535] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Sep 23 15:26:30 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:26:30.655155 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae4f260-f7ae-40f4-beb8-68a569a38a48 admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:26:30.665062 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae4f260-f7ae-40f4-beb8-68a569a38a48 admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:26:30.669691 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75f22f9-3ac8-4c94-abef-0f2b8206ffbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:26:30.671146 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75f22f9-3ac8-4c94-abef-0f2b8206ffbe admin admin] REQUEST_METHOD: `PUT` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:30.671487 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75f22f9-3ac8-4c94-abef-0f2b8206ffbe admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:30.671938 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75f22f9-3ac8-4c94-abef-0f2b8206ffbe admin admin] PATH_INFO: `/v3/projects/5e95705a25404ccb830036cf02e07a52/users/9df2962e017847539a96d9a00a2da16a/roles/8e2919e79df040e8926f21607818b58c` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:30.687004 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75f22f9-3ac8-4c94-abef-0f2b8206ffbe admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e95705a25404ccb830036cf02e07a52, user_id=9df2962e017847539a96d9a00a2da16a, role_id=8e2919e79df040e8926f21607818b58c)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:26:30.689729 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75f22f9-3ac8-4c94-abef-0f2b8206ffbe admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:26:30.750968 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1282/2536] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Sep 23 15:26:30 2022] POST /identity/v3/users => generated 289 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 23 15:26:30.755303 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184a44b5-f161-4d4a-9fea-4926dc19a457 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:26:30.765396 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184a44b5-f161-4d4a-9fea-4926dc19a457 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:26:30.770082 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb0cbe2-f9a7-4dba-90db-b1ceae95a76c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:26:30.770742 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb0cbe2-f9a7-4dba-90db-b1ceae95a76c admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:30.770989 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb0cbe2-f9a7-4dba-90db-b1ceae95a76c admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:30.771623 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb0cbe2-f9a7-4dba-90db-b1ceae95a76c admin admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:30.772492 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb0cbe2-f9a7-4dba-90db-b1ceae95a76c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:26:30.774619 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb0cbe2-f9a7-4dba-90db-b1ceae95a76c admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:26:30.780499 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1255/2537] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Sep 23 15:26:30 2022] PUT /identity/v3/projects/5e95705a25404ccb830036cf02e07a52/users/9df2962e017847539a96d9a00a2da16a/roles/8e2919e79df040e8926f21607818b58c => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:26:30.786707 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1283/2538] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Sep 23 15:26:30 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 23 15:26:30.788678 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75f22f9-3ac8-4c94-abef-0f2b8206ffbe admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:26:30.792081 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb0cbe2-f9a7-4dba-90db-b1ceae95a76c admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:26:30.803635 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb0cbe2-f9a7-4dba-90db-b1ceae95a76c admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:26:30.865452 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75f22f9-3ac8-4c94-abef-0f2b8206ffbe admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:26:30.868881 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51575c8b-3d54-4609-9606-a0f41bb3440e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:26:30.869501 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51575c8b-3d54-4609-9606-a0f41bb3440e admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:30.869787 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51575c8b-3d54-4609-9606-a0f41bb3440e admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:30.870065 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51575c8b-3d54-4609-9606-a0f41bb3440e admin admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:30.870713 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51575c8b-3d54-4609-9606-a0f41bb3440e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:26:30.871881 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a489df92-1084-4897-b39b-6d13fd47bdcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:26:30.872617 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51575c8b-3d54-4609-9606-a0f41bb3440e admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:26:30.872933 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a489df92-1084-4897-b39b-6d13fd47bdcc admin admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:30.873562 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a489df92-1084-4897-b39b-6d13fd47bdcc admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:30.873853 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a489df92-1084-4897-b39b-6d13fd47bdcc admin admin] PATH_INFO: `/v3/roles` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:30.874763 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a489df92-1084-4897-b39b-6d13fd47bdcc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:26:30.877165 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a489df92-1084-4897-b39b-6d13fd47bdcc admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:26:30.889221 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1284/2539] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Sep 23 15:26:30 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 23 15:26:30.889420 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1256/2540] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Sep 23 15:26:30 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 23 15:26:30.893540 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a489df92-1084-4897-b39b-6d13fd47bdcc admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:26:30.893685 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51575c8b-3d54-4609-9606-a0f41bb3440e admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:26:30.905084 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a489df92-1084-4897-b39b-6d13fd47bdcc admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:26:30.905496 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51575c8b-3d54-4609-9606-a0f41bb3440e admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:26:30.910692 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:26:30.912524 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7a2f30-16c4-4e84-aa14-e53b713dbcda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:26:30.913370 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7a2f30-16c4-4e84-aa14-e53b713dbcda admin admin] REQUEST_METHOD: `PUT` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:30.913717 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7a2f30-16c4-4e84-aa14-e53b713dbcda admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:30.921560 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9c7a65-8616-4f61-9490-1533a4be232d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:26:30.921744 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7a2f30-16c4-4e84-aa14-e53b713dbcda admin admin] PATH_INFO: `/v3/projects/5e95705a25404ccb830036cf02e07a52/users/9df2962e017847539a96d9a00a2da16a/roles/8e2919e79df040e8926f21607818b58c` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:30.921854 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9c7a65-8616-4f61-9490-1533a4be232d admin admin] REQUEST_METHOD: `PUT` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:30.921854 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9c7a65-8616-4f61-9490-1533a4be232d admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:30.921854 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9c7a65-8616-4f61-9490-1533a4be232d admin admin] PATH_INFO: `/v3/projects/d86aa99c129b476980f17821cd30ef30/users/1cb120cee6bf4ef4b7da15735f5f77e1/roles/af257be2dee54d6b942f58650196582c` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:30.922140 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7a2f30-16c4-4e84-aa14-e53b713dbcda admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e95705a25404ccb830036cf02e07a52, user_id=9df2962e017847539a96d9a00a2da16a, role_id=8e2919e79df040e8926f21607818b58c)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:26:30.922140 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7a2f30-16c4-4e84-aa14-e53b713dbcda admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:26:30.934927 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1257/2541] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Sep 23 15:26:30 2022] PUT /identity/v3/projects/5e95705a25404ccb830036cf02e07a52/users/9df2962e017847539a96d9a00a2da16a/roles/8e2919e79df040e8926f21607818b58c => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 15:26:30.935340 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9c7a65-8616-4f61-9490-1533a4be232d admin admin] RBAC: Authorizing `identity:create_grant(project_id=d86aa99c129b476980f17821cd30ef30, user_id=1cb120cee6bf4ef4b7da15735f5f77e1, role_id=af257be2dee54d6b942f58650196582c)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:26:30.938886 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9c7a65-8616-4f61-9490-1533a4be232d admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:26:30.945523 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af2974c-fea4-458d-b31d-dd4f1e96afcf None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:30.945872 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af2974c-fea4-458d-b31d-dd4f1e96afcf None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:30.950036 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af2974c-fea4-458d-b31d-dd4f1e96afcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:30.976095 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-2af2974c-fea4-458d-b31d-dd4f1e96afcf None None] MFA Rules not processed for user `9df2962e017847539a96d9a00a2da16a`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:26:30.977403 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-2af2974c-fea4-458d-b31d-dd4f1e96afcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:26:31.050425 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1258/2542] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:26:30 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:26:31.069018 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af2974c-fea4-458d-b31d-dd4f1e96afcf None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:26:31.070537 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-2af2974c-fea4-458d-b31d-dd4f1e96afcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:26:31.076882 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1285/2543] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Sep 23 15:26:30 2022] PUT /identity/v3/projects/d86aa99c129b476980f17821cd30ef30/users/1cb120cee6bf4ef4b7da15735f5f77e1/roles/af257be2dee54d6b942f58650196582c => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:26:31.085374 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b83bf0-df42-41bc-9aca-bc03d84ba58e None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:31.085374 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b83bf0-df42-41bc-9aca-bc03d84ba58e None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:31.085374 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b83bf0-df42-41bc-9aca-bc03d84ba58e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:31.103611 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-63b83bf0-df42-41bc-9aca-bc03d84ba58e None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:26:31.104872 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-63b83bf0-df42-41bc-9aca-bc03d84ba58e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:26:31.158062 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af2974c-fea4-458d-b31d-dd4f1e96afcf None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:26:31.172458 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1286/2544] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:26:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 15:26:31.182378 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b83bf0-df42-41bc-9aca-bc03d84ba58e None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:26:31.183785 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-63b83bf0-df42-41bc-9aca-bc03d84ba58e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:26:31.225817 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3875892-e167-4f0d-b9dd-959611810653 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feca6278be904ef8b0dc580a67e477f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:26:31.225817 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3875892-e167-4f0d-b9dd-959611810653 service cinder] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:31.225817 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3875892-e167-4f0d-b9dd-959611810653 service cinder] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:31.225817 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3875892-e167-4f0d-b9dd-959611810653 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:31.256379 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b83bf0-df42-41bc-9aca-bc03d84ba58e None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:26:31.260547 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314ae378-064f-43bb-94ae-91b09d4cc293 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd259178570864ffeb936ee10dfec6b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:26:31.261230 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314ae378-064f-43bb-94ae-91b09d4cc293 service neutron] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:31.269028 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314ae378-064f-43bb-94ae-91b09d4cc293 service neutron] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:31.269028 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314ae378-064f-43bb-94ae-91b09d4cc293 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:31.271908 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314ae378-064f-43bb-94ae-91b09d4cc293 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:26:31.273419 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314ae378-064f-43bb-94ae-91b09d4cc293 service neutron] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:26:31.284535 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1287/2545] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:26:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 23 15:26:31.308067 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3875892-e167-4f0d-b9dd-959611810653 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:26:31.309554 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3875892-e167-4f0d-b9dd-959611810653 service cinder] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:26:31.320958 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1259/2546] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:26:31 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 252 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:26:31.418738 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:26:31.603490 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4f5226-c966-4dfe-9492-6532e4e6a540 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:31.603914 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4f5226-c966-4dfe-9492-6532e4e6a540 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:31.604290 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4f5226-c966-4dfe-9492-6532e4e6a540 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:31.638514 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-2b4f5226-c966-4dfe-9492-6532e4e6a540 None None] MFA Rules not processed for user `68206a6b4f244d45bf7bf1504396e430`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:26:31.641365 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-2b4f5226-c966-4dfe-9492-6532e4e6a540 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:26:31.703522 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1288/2547] 10.222.0.55 () {58 vars in 1009 bytes} [Fri Sep 23 15:26:31 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:26:31.716406 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3875892-e167-4f0d-b9dd-959611810653 service cinder] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:26:31.718615 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-b3875892-e167-4f0d-b9dd-959611810653 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:26:31.795662 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3875892-e167-4f0d-b9dd-959611810653 service cinder] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:26:31.802470 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2255ba02-55ca-41db-bb20-825786b70162 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2228c0b49a0249bbaf524a8c4e9e108b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:26:31.802470 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2255ba02-55ca-41db-bb20-825786b70162 service swift] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:31.802470 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2255ba02-55ca-41db-bb20-825786b70162 service swift] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:31.802470 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2255ba02-55ca-41db-bb20-825786b70162 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:31.811377 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2255ba02-55ca-41db-bb20-825786b70162 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:26:31.813194 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2255ba02-55ca-41db-bb20-825786b70162 service swift] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:26:31.821699 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1260/2548] 10.222.0.55 () {60 vars in 1384 bytes} [Fri Sep 23 15:26:31 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 23 15:26:32.006430 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:26:32.198803 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:26:32.917884 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4f5226-c966-4dfe-9492-6532e4e6a540 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:26:32.920490 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-2b4f5226-c966-4dfe-9492-6532e4e6a540 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:26:32.936895 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4f5226-c966-4dfe-9492-6532e4e6a540 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:26:32.941240 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5995ba2f-3f30-488a-9d89-dadd142d0f26 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feca6278be904ef8b0dc580a67e477f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:26:32.942135 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5995ba2f-3f30-488a-9d89-dadd142d0f26 service cinder] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:32.942489 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5995ba2f-3f30-488a-9d89-dadd142d0f26 service cinder] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:32.942836 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5995ba2f-3f30-488a-9d89-dadd142d0f26 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:32.944809 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-5995ba2f-3f30-488a-9d89-dadd142d0f26 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:26:33.021115 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5995ba2f-3f30-488a-9d89-dadd142d0f26 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:26:33.022767 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5995ba2f-3f30-488a-9d89-dadd142d0f26 service cinder] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:26:33.053525 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1289/2549] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:26:32 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:26:39.364296 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2255ba02-55ca-41db-bb20-825786b70162 service swift] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:26:39.382242 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2255ba02-55ca-41db-bb20-825786b70162 service swift] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:26:39.399235 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8a5e6b-6298-4279-8d70-c96bcb263983 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feca6278be904ef8b0dc580a67e477f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:26:39.399968 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8a5e6b-6298-4279-8d70-c96bcb263983 service cinder] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:39.400276 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8a5e6b-6298-4279-8d70-c96bcb263983 service cinder] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:39.400554 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8a5e6b-6298-4279-8d70-c96bcb263983 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:39.493040 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecba270-720a-432f-8d8c-1cd26226a484 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:39.496407 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecba270-720a-432f-8d8c-1cd26226a484 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:39.497796 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecba270-720a-432f-8d8c-1cd26226a484 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:39.544590 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8a5e6b-6298-4279-8d70-c96bcb263983 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:26:39.544590 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8a5e6b-6298-4279-8d70-c96bcb263983 service cinder] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:26:39.549675 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-4ecba270-720a-432f-8d8c-1cd26226a484 None None] MFA Rules not processed for user `1cb120cee6bf4ef4b7da15735f5f77e1`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:26:39.551542 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-4ecba270-720a-432f-8d8c-1cd26226a484 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:26:39.552167 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1261/2550] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:26:39 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:26:39.572996 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:26:39.663172 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1290/2551] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:26:39 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 15:26:39.665342 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8a5e6b-6298-4279-8d70-c96bcb263983 service cinder] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:26:39.681215 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8a5e6b-6298-4279-8d70-c96bcb263983 service cinder] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:26:39.691940 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e902826e-4f14-49b2-89da-95004d03c741 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:26:39.693539 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e902826e-4f14-49b2-89da-95004d03c741 admin admin] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:39.693935 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e902826e-4f14-49b2-89da-95004d03c741 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:39.694304 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e902826e-4f14-49b2-89da-95004d03c741 admin admin] PATH_INFO: `/v3/projects` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:39.696419 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e902826e-4f14-49b2-89da-95004d03c741 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:26:39.699163 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e902826e-4f14-49b2-89da-95004d03c741 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:26:39.722903 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:26:40.073039 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1262/2552] 10.222.0.55 () {60 vars in 1153 bytes} [Fri Sep 23 15:26:39 2022] POST /identity/v3/projects => generated 375 bytes in 410 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:26:40.082865 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecba270-720a-432f-8d8c-1cd26226a484 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:26:40.094458 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecba270-720a-432f-8d8c-1cd26226a484 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:26:40.154885 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efaf59e2-2358-4df4-9f89-f1336a5f014f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:26:40.155554 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaf59e2-2358-4df4-9f89-f1336a5f014f admin admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:40.155838 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaf59e2-2358-4df4-9f89-f1336a5f014f admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:40.156165 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaf59e2-2358-4df4-9f89-f1336a5f014f admin admin] PATH_INFO: `/v3/users` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:40.156822 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efaf59e2-2358-4df4-9f89-f1336a5f014f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:26:40.158714 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efaf59e2-2358-4df4-9f89-f1336a5f014f admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:26:40.370526 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1291/2553] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Sep 23 15:26:40 2022] POST /identity/v3/users => generated 294 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:26:40.371518 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e902826e-4f14-49b2-89da-95004d03c741 admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:26:40.382625 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e902826e-4f14-49b2-89da-95004d03c741 admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:26:40.387339 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4e25ab-7549-4fb1-9dc3-62fc65bcc338 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:26:40.387944 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4e25ab-7549-4fb1-9dc3-62fc65bcc338 admin admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:40.388263 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4e25ab-7549-4fb1-9dc3-62fc65bcc338 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:40.388542 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4e25ab-7549-4fb1-9dc3-62fc65bcc338 admin admin] PATH_INFO: `/v3/roles` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:40.389191 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4e25ab-7549-4fb1-9dc3-62fc65bcc338 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:26:40.390974 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4e25ab-7549-4fb1-9dc3-62fc65bcc338 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:26:40.402395 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1263/2554] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Sep 23 15:26:40 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 23 15:26:40.407253 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efaf59e2-2358-4df4-9f89-f1336a5f014f admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:26:40.418052 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efaf59e2-2358-4df4-9f89-f1336a5f014f admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:26:40.427287 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a520c4-4a4d-4b86-8cf2-a10e29a69d95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:26:40.428185 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a520c4-4a4d-4b86-8cf2-a10e29a69d95 admin admin] REQUEST_METHOD: `PUT` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:40.428562 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a520c4-4a4d-4b86-8cf2-a10e29a69d95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:40.428902 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a520c4-4a4d-4b86-8cf2-a10e29a69d95 admin admin] PATH_INFO: `/v3/projects/50f11b8c3f7949debec23f52ea06b70c/users/b401a9f68da44913aef7366b1e69af65/roles/8e2919e79df040e8926f21607818b58c` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:40.454757 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a520c4-4a4d-4b86-8cf2-a10e29a69d95 admin admin] RBAC: Authorizing `identity:create_grant(project_id=50f11b8c3f7949debec23f52ea06b70c, user_id=b401a9f68da44913aef7366b1e69af65, role_id=8e2919e79df040e8926f21607818b58c)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:26:40.457834 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a520c4-4a4d-4b86-8cf2-a10e29a69d95 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:26:40.535195 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1292/2555] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Sep 23 15:26:40 2022] PUT /identity/v3/projects/50f11b8c3f7949debec23f52ea06b70c/users/b401a9f68da44913aef7366b1e69af65/roles/8e2919e79df040e8926f21607818b58c => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:26:40.539293 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4e25ab-7549-4fb1-9dc3-62fc65bcc338 admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:26:40.552546 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4e25ab-7549-4fb1-9dc3-62fc65bcc338 admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:26:40.649394 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ed5dac-41cc-4bba-9d77-73466e8b5053 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:26:40.650909 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ed5dac-41cc-4bba-9d77-73466e8b5053 admin admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:40.656431 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ed5dac-41cc-4bba-9d77-73466e8b5053 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:40.656431 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ed5dac-41cc-4bba-9d77-73466e8b5053 admin admin] PATH_INFO: `/v3/roles` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:40.656431 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ed5dac-41cc-4bba-9d77-73466e8b5053 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:26:40.656431 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ed5dac-41cc-4bba-9d77-73466e8b5053 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:26:40.673987 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1264/2556] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Sep 23 15:26:40 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:26:40.677977 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a520c4-4a4d-4b86-8cf2-a10e29a69d95 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:26:40.691778 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a520c4-4a4d-4b86-8cf2-a10e29a69d95 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:26:40.699809 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4b7aeb-8ff4-48f4-870c-8c6cb68aacd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:26:40.700621 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4b7aeb-8ff4-48f4-870c-8c6cb68aacd6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:40.701552 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4b7aeb-8ff4-48f4-870c-8c6cb68aacd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:40.701928 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4b7aeb-8ff4-48f4-870c-8c6cb68aacd6 admin admin] PATH_INFO: `/v3/projects/50f11b8c3f7949debec23f52ea06b70c/users/b401a9f68da44913aef7366b1e69af65/roles/8e2919e79df040e8926f21607818b58c` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:40.704661 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4b7aeb-8ff4-48f4-870c-8c6cb68aacd6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=50f11b8c3f7949debec23f52ea06b70c, user_id=b401a9f68da44913aef7366b1e69af65, role_id=8e2919e79df040e8926f21607818b58c)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:26:40.707739 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4b7aeb-8ff4-48f4-870c-8c6cb68aacd6 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:26:40.724797 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1293/2557] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Sep 23 15:26:40 2022] PUT /identity/v3/projects/50f11b8c3f7949debec23f52ea06b70c/users/b401a9f68da44913aef7366b1e69af65/roles/8e2919e79df040e8926f21607818b58c => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 15:26:46.439254 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ed5dac-41cc-4bba-9d77-73466e8b5053 admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:26:46.455547 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ed5dac-41cc-4bba-9d77-73466e8b5053 admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:26:46.528684 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8156b8a5-7d09-4201-bf04-a29a69b846a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:26:46.530056 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8156b8a5-7d09-4201-bf04-a29a69b846a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:46.530431 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8156b8a5-7d09-4201-bf04-a29a69b846a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:46.530789 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8156b8a5-7d09-4201-bf04-a29a69b846a3 admin admin] PATH_INFO: `/v3/users/32d96158cee44725b1458a9d3b1190bb` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:46.532046 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8156b8a5-7d09-4201-bf04-a29a69b846a3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=32d96158cee44725b1458a9d3b1190bb)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:26:46.534174 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8156b8a5-7d09-4201-bf04-a29a69b846a3 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:26:46.771373 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-8156b8a5-7d09-4201-bf04-a29a69b846a3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '32d96158cee44725b1458a9d3b1190bb'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:46.790708 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-8156b8a5-7d09-4201-bf04-a29a69b846a3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '32d96158cee44725b1458a9d3b1190bb'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:46.799732 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-8156b8a5-7d09-4201-bf04-a29a69b846a3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '32d96158cee44725b1458a9d3b1190bb'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:46.916903 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-8156b8a5-7d09-4201-bf04-a29a69b846a3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '32d96158cee44725b1458a9d3b1190bb'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:46.918914 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-8156b8a5-7d09-4201-bf04-a29a69b846a3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '32d96158cee44725b1458a9d3b1190bb'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:46.922181 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:INSERT=2,keystone:SELECT=88,keystone:DELETE=10 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:26:46.933123 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1265/2558] 10.222.0.55 () {58 vars in 1224 bytes} [Fri Sep 23 15:26:46 2022] DELETE /identity/v3/users/32d96158cee44725b1458a9d3b1190bb => generated 0 bytes in 491 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:26:46.934518 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8b4df5-f1e6-45ac-a607-e11a9bbc847a None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:46.934984 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8b4df5-f1e6-45ac-a607-e11a9bbc847a None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:46.935381 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8b4df5-f1e6-45ac-a607-e11a9bbc847a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:46.961275 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-bb8b4df5-f1e6-45ac-a607-e11a9bbc847a None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:26:46.962825 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-bb8b4df5-f1e6-45ac-a607-e11a9bbc847a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:26:47.044709 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1294/2559] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:26:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:26:47.059584 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8156b8a5-7d09-4201-bf04-a29a69b846a3 admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:26:47.061718 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-8156b8a5-7d09-4201-bf04-a29a69b846a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:26:47.153355 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8156b8a5-7d09-4201-bf04-a29a69b846a3 admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:26:47.161857 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfc6910-e484-4c56-bf05-3eb1af07d6eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd259178570864ffeb936ee10dfec6b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:26:47.162493 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfc6910-e484-4c56-bf05-3eb1af07d6eb service neutron] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:47.162773 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfc6910-e484-4c56-bf05-3eb1af07d6eb service neutron] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:47.163073 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfc6910-e484-4c56-bf05-3eb1af07d6eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:47.181601 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfc6910-e484-4c56-bf05-3eb1af07d6eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:26:47.188280 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfc6910-e484-4c56-bf05-3eb1af07d6eb service neutron] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:26:47.197012 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1266/2560] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:26:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:26:47.743512 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8b4df5-f1e6-45ac-a607-e11a9bbc847a None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:26:47.749991 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-bb8b4df5-f1e6-45ac-a607-e11a9bbc847a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:26:47.759656 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8b4df5-f1e6-45ac-a607-e11a9bbc847a None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:26:47.764608 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309a6bf2-ecb9-4015-9dcf-b21c79f08d2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:26:47.765376 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309a6bf2-ecb9-4015-9dcf-b21c79f08d2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:47.765990 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309a6bf2-ecb9-4015-9dcf-b21c79f08d2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:47.766126 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309a6bf2-ecb9-4015-9dcf-b21c79f08d2d admin admin] PATH_INFO: `/v3/projects/fc9693b3782346509e69e05f8a072615` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:47.767745 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309a6bf2-ecb9-4015-9dcf-b21c79f08d2d admin admin] RBAC: Authorizing `identity:delete_project(project_id=fc9693b3782346509e69e05f8a072615)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:26:47.769591 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309a6bf2-ecb9-4015-9dcf-b21c79f08d2d admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:26:47.772654 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:26:47.936838 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-309a6bf2-ecb9-4015-9dcf-b21c79f08d2d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fc9693b3782346509e69e05f8a072615', 'request_id': 'req-309a6bf2-ecb9-4015-9dcf-b21c79f08d2d'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:47.943223 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-309a6bf2-ecb9-4015-9dcf-b21c79f08d2d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fc9693b3782346509e69e05f8a072615', 'request_id': 'req-309a6bf2-ecb9-4015-9dcf-b21c79f08d2d'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:47.948896 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-309a6bf2-ecb9-4015-9dcf-b21c79f08d2d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fc9693b3782346509e69e05f8a072615', 'request_id': 'req-309a6bf2-ecb9-4015-9dcf-b21c79f08d2d'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:48.120317 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-309a6bf2-ecb9-4015-9dcf-b21c79f08d2d admin admin] The token cache is being invalidate because project fc9693b3782346509e69e05f8a072615 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81994) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 15:26:48.127772 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-309a6bf2-ecb9-4015-9dcf-b21c79f08d2d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:48.127772 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-309a6bf2-ecb9-4015-9dcf-b21c79f08d2d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:48.127772 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1295/2561] 10.222.0.55 () {58 vars in 1233 bytes} [Fri Sep 23 15:26:47 2022] DELETE /identity/v3/projects/fc9693b3782346509e69e05f8a072615 => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:26:48.130285 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfc6910-e484-4c56-bf05-3eb1af07d6eb service neutron] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:26:48.142884 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfc6910-e484-4c56-bf05-3eb1af07d6eb service neutron] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:26:48.227662 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfaafcd6-c16b-4bbc-b42f-83a24972253b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:26:48.228329 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaafcd6-c16b-4bbc-b42f-83a24972253b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:48.228602 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaafcd6-c16b-4bbc-b42f-83a24972253b admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:48.228879 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaafcd6-c16b-4bbc-b42f-83a24972253b admin admin] PATH_INFO: `/v3/users/b15cbfe9a3644c7aacc5ebcc77bd5fdc` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:48.230083 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfaafcd6-c16b-4bbc-b42f-83a24972253b admin admin] RBAC: Authorizing `identity:delete_user(user_id=b15cbfe9a3644c7aacc5ebcc77bd5fdc)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:26:48.232214 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfaafcd6-c16b-4bbc-b42f-83a24972253b admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:26:48.523458 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-cfaafcd6-c16b-4bbc-b42f-83a24972253b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b15cbfe9a3644c7aacc5ebcc77bd5fdc'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:48.541646 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-cfaafcd6-c16b-4bbc-b42f-83a24972253b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b15cbfe9a3644c7aacc5ebcc77bd5fdc'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:48.550733 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-cfaafcd6-c16b-4bbc-b42f-83a24972253b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b15cbfe9a3644c7aacc5ebcc77bd5fdc'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:48.629709 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-cfaafcd6-c16b-4bbc-b42f-83a24972253b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b15cbfe9a3644c7aacc5ebcc77bd5fdc'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:48.630354 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-cfaafcd6-c16b-4bbc-b42f-83a24972253b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b15cbfe9a3644c7aacc5ebcc77bd5fdc'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:48.640708 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1267/2562] 10.222.0.55 () {58 vars in 1224 bytes} [Fri Sep 23 15:26:48 2022] DELETE /identity/v3/users/b15cbfe9a3644c7aacc5ebcc77bd5fdc => generated 0 bytes in 509 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 15:26:49.197607 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309a6bf2-ecb9-4015-9dcf-b21c79f08d2d admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:26:49.197607 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-309a6bf2-ecb9-4015-9dcf-b21c79f08d2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:26:49.221144 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309a6bf2-ecb9-4015-9dcf-b21c79f08d2d admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:26:49.366015 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9388af-820a-4071-8022-afd7fb3703b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:26:49.366680 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9388af-820a-4071-8022-afd7fb3703b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:49.366961 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9388af-820a-4071-8022-afd7fb3703b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:49.367235 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9388af-820a-4071-8022-afd7fb3703b8 admin admin] PATH_INFO: `/v3/projects/442f001261ea48be83436bfc65116e65` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:49.368433 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9388af-820a-4071-8022-afd7fb3703b8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=442f001261ea48be83436bfc65116e65)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:26:49.370398 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9388af-820a-4071-8022-afd7fb3703b8 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:26:49.556462 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-1d9388af-820a-4071-8022-afd7fb3703b8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '442f001261ea48be83436bfc65116e65', 'request_id': 'req-1d9388af-820a-4071-8022-afd7fb3703b8'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:49.569574 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-1d9388af-820a-4071-8022-afd7fb3703b8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '442f001261ea48be83436bfc65116e65', 'request_id': 'req-1d9388af-820a-4071-8022-afd7fb3703b8'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:49.580832 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-1d9388af-820a-4071-8022-afd7fb3703b8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '442f001261ea48be83436bfc65116e65', 'request_id': 'req-1d9388af-820a-4071-8022-afd7fb3703b8'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:49.608485 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1deb3e10-c2a3-4317-afc2-4891e2356a89 None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:49.609028 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1deb3e10-c2a3-4317-afc2-4891e2356a89 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:49.609127 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1deb3e10-c2a3-4317-afc2-4891e2356a89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:49.648966 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-1deb3e10-c2a3-4317-afc2-4891e2356a89 None None] MFA Rules not processed for user `b401a9f68da44913aef7366b1e69af65`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:26:49.650748 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-1deb3e10-c2a3-4317-afc2-4891e2356a89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:26:49.664261 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:26:49.686878 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-1d9388af-820a-4071-8022-afd7fb3703b8 admin admin] The token cache is being invalidate because project 442f001261ea48be83436bfc65116e65 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81994) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 15:26:49.691887 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-1d9388af-820a-4071-8022-afd7fb3703b8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:49.691887 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-1d9388af-820a-4071-8022-afd7fb3703b8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:49.691887 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1296/2563] 10.222.0.55 () {58 vars in 1233 bytes} [Fri Sep 23 15:26:49 2022] DELETE /identity/v3/projects/442f001261ea48be83436bfc65116e65 => generated 0 bytes in 499 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:26:49.694323 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9388af-820a-4071-8022-afd7fb3703b8 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:26:49.697755 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-1d9388af-820a-4071-8022-afd7fb3703b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:26:49.732947 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:26:49.756595 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1268/2564] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:26:49 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 15:26:49.764297 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1deb3e10-c2a3-4317-afc2-4891e2356a89 None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:26:49.766370 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-1deb3e10-c2a3-4317-afc2-4891e2356a89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:26:49.778138 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9388af-820a-4071-8022-afd7fb3703b8 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:26:49.784019 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d914ab-9c22-493f-ac41-cf6499183780 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:26:49.784702 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d914ab-9c22-493f-ac41-cf6499183780 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:49.784987 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d914ab-9c22-493f-ac41-cf6499183780 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:49.785295 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d914ab-9c22-493f-ac41-cf6499183780 admin admin] PATH_INFO: `/v3/users/9df2962e017847539a96d9a00a2da16a` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:49.786452 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d914ab-9c22-493f-ac41-cf6499183780 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9df2962e017847539a96d9a00a2da16a)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:26:49.788544 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d914ab-9c22-493f-ac41-cf6499183780 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:26:49.844308 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1deb3e10-c2a3-4317-afc2-4891e2356a89 None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:26:49.849913 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3103ba28-a38b-4ce6-b043-7d8e5c01e6dd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3c6c7a1325a47be924268e1aa1a7a18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:26:49.850536 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3103ba28-a38b-4ce6-b043-7d8e5c01e6dd service nova] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:49.850816 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3103ba28-a38b-4ce6-b043-7d8e5c01e6dd service nova] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:49.851092 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3103ba28-a38b-4ce6-b043-7d8e5c01e6dd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:49.852863 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-3103ba28-a38b-4ce6-b043-7d8e5c01e6dd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:26:49.916811 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3103ba28-a38b-4ce6-b043-7d8e5c01e6dd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:26:49.918431 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3103ba28-a38b-4ce6-b043-7d8e5c01e6dd service nova] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:26:49.927856 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1269/2565] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:26:49 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 23 15:26:49.967873 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-a3d914ab-9c22-493f-ac41-cf6499183780 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9df2962e017847539a96d9a00a2da16a'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:49.980660 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-a3d914ab-9c22-493f-ac41-cf6499183780 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9df2962e017847539a96d9a00a2da16a'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:49.988254 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-a3d914ab-9c22-493f-ac41-cf6499183780 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9df2962e017847539a96d9a00a2da16a'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:50.068740 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-a3d914ab-9c22-493f-ac41-cf6499183780 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9df2962e017847539a96d9a00a2da16a'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:50.069393 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-a3d914ab-9c22-493f-ac41-cf6499183780 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9df2962e017847539a96d9a00a2da16a'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:50.077581 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1297/2566] 10.222.0.55 () {58 vars in 1224 bytes} [Fri Sep 23 15:26:49 2022] DELETE /identity/v3/users/9df2962e017847539a96d9a00a2da16a => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:26:50.585109 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3103ba28-a38b-4ce6-b043-7d8e5c01e6dd service nova] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:26:50.616116 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3103ba28-a38b-4ce6-b043-7d8e5c01e6dd service nova] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:26:50.660682 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ec0912-7e99-412f-9db3-5c3f23944b88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:26:50.668831 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ec0912-7e99-412f-9db3-5c3f23944b88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:50.668831 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ec0912-7e99-412f-9db3-5c3f23944b88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:50.668831 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ec0912-7e99-412f-9db3-5c3f23944b88 admin admin] PATH_INFO: `/v3/projects/5e95705a25404ccb830036cf02e07a52` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:50.668831 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ec0912-7e99-412f-9db3-5c3f23944b88 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5e95705a25404ccb830036cf02e07a52)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:26:50.668831 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ec0912-7e99-412f-9db3-5c3f23944b88 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:26:50.787630 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:26:50.809395 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-c5ec0912-7e99-412f-9db3-5c3f23944b88 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5e95705a25404ccb830036cf02e07a52', 'request_id': 'req-c5ec0912-7e99-412f-9db3-5c3f23944b88'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:50.819314 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-c5ec0912-7e99-412f-9db3-5c3f23944b88 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5e95705a25404ccb830036cf02e07a52', 'request_id': 'req-c5ec0912-7e99-412f-9db3-5c3f23944b88'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:50.943049 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-c5ec0912-7e99-412f-9db3-5c3f23944b88 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5e95705a25404ccb830036cf02e07a52', 'request_id': 'req-c5ec0912-7e99-412f-9db3-5c3f23944b88'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:50.956397 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-c5ec0912-7e99-412f-9db3-5c3f23944b88 admin admin] The token cache is being invalidate because project 5e95705a25404ccb830036cf02e07a52 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81993) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 15:26:50.956686 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-c5ec0912-7e99-412f-9db3-5c3f23944b88 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:50.957346 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-c5ec0912-7e99-412f-9db3-5c3f23944b88 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:50.958948 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1270/2567] 10.222.0.55 () {58 vars in 1233 bytes} [Fri Sep 23 15:26:50 2022] DELETE /identity/v3/projects/5e95705a25404ccb830036cf02e07a52 => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:26:50.965138 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad47b5a-9dc9-42a4-9711-57ab3f3236b5 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:50.966429 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad47b5a-9dc9-42a4-9711-57ab3f3236b5 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:50.966716 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad47b5a-9dc9-42a4-9711-57ab3f3236b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:50.988101 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-6ad47b5a-9dc9-42a4-9711-57ab3f3236b5 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:26:50.989369 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-6ad47b5a-9dc9-42a4-9711-57ab3f3236b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:26:51.065780 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1298/2568] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:26:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:26:51.077020 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527d7215-bc58-4c61-8d5a-dbc9863bf8e4 None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:51.077356 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527d7215-bc58-4c61-8d5a-dbc9863bf8e4 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:51.077640 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527d7215-bc58-4c61-8d5a-dbc9863bf8e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:51.110270 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-527d7215-bc58-4c61-8d5a-dbc9863bf8e4 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:26:51.111667 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-527d7215-bc58-4c61-8d5a-dbc9863bf8e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:26:51.121063 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1271/2569] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:26:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:26:51.126082 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad47b5a-9dc9-42a4-9711-57ab3f3236b5 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:26:51.136495 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad47b5a-9dc9-42a4-9711-57ab3f3236b5 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:26:51.143608 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d2ed84-2122-4b08-9aac-c765f65347b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:26:51.160207 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d2ed84-2122-4b08-9aac-c765f65347b8 admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:51.160207 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d2ed84-2122-4b08-9aac-c765f65347b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:51.160207 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d2ed84-2122-4b08-9aac-c765f65347b8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:51.160207 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d2ed84-2122-4b08-9aac-c765f65347b8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:26:51.160207 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d2ed84-2122-4b08-9aac-c765f65347b8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:26:51.160207 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d2ed84-2122-4b08-9aac-c765f65347b8 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:26:51.166464 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1299/2570] 10.222.0.55 () {58 vars in 1153 bytes} [Fri Sep 23 15:26:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 15:26:51.171393 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527d7215-bc58-4c61-8d5a-dbc9863bf8e4 None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:26:51.181917 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527d7215-bc58-4c61-8d5a-dbc9863bf8e4 None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:26:51.187493 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ded3e4-5d93-4fc3-8726-f5cfbf5c3cbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:26:51.188174 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ded3e4-5d93-4fc3-8726-f5cfbf5c3cbb admin admin] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:51.188455 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ded3e4-5d93-4fc3-8726-f5cfbf5c3cbb admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:51.188728 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ded3e4-5d93-4fc3-8726-f5cfbf5c3cbb admin admin] PATH_INFO: `/v3/projects` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:51.189372 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ded3e4-5d93-4fc3-8726-f5cfbf5c3cbb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:26:51.191276 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ded3e4-5d93-4fc3-8726-f5cfbf5c3cbb admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:26:51.397054 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1272/2571] 10.222.0.55 () {60 vars in 1153 bytes} [Fri Sep 23 15:26:51 2022] POST /identity/v3/projects => generated 391 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:26:51.400411 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d2ed84-2122-4b08-9aac-c765f65347b8 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:26:51.410936 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d2ed84-2122-4b08-9aac-c765f65347b8 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:26:51.441126 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:26:51.476416 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f7ba6f-2a2b-4a32-940b-1803bc21df0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:26:51.477396 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f7ba6f-2a2b-4a32-940b-1803bc21df0f admin admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:51.477673 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f7ba6f-2a2b-4a32-940b-1803bc21df0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:51.477941 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f7ba6f-2a2b-4a32-940b-1803bc21df0f admin admin] PATH_INFO: `/v3/users` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:51.478578 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f7ba6f-2a2b-4a32-940b-1803bc21df0f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:26:51.480784 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f7ba6f-2a2b-4a32-940b-1803bc21df0f admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:26:51.637204 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1300/2572] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Sep 23 15:26:51 2022] POST /identity/v3/users => generated 296 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 23 15:26:51.641299 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ded3e4-5d93-4fc3-8726-f5cfbf5c3cbb admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:26:51.656242 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ded3e4-5d93-4fc3-8726-f5cfbf5c3cbb admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:26:51.665838 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2bd51a-4088-4608-abb9-d5664db60ce5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:26:51.666467 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2bd51a-4088-4608-abb9-d5664db60ce5 admin admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:51.666753 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2bd51a-4088-4608-abb9-d5664db60ce5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:51.667087 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2bd51a-4088-4608-abb9-d5664db60ce5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:51.676004 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2bd51a-4088-4608-abb9-d5664db60ce5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:26:51.676004 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2bd51a-4088-4608-abb9-d5664db60ce5 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:26:51.681047 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1273/2573] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Sep 23 15:26:51 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:26:51.685515 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f7ba6f-2a2b-4a32-940b-1803bc21df0f admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:26:51.696427 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f7ba6f-2a2b-4a32-940b-1803bc21df0f admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:26:51.702012 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06893e32-acac-44de-9ff5-cefb4f9a55fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:26:51.705486 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06893e32-acac-44de-9ff5-cefb4f9a55fd admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:51.705486 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06893e32-acac-44de-9ff5-cefb4f9a55fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:51.705486 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06893e32-acac-44de-9ff5-cefb4f9a55fd admin admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:51.705486 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06893e32-acac-44de-9ff5-cefb4f9a55fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:26:51.705936 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06893e32-acac-44de-9ff5-cefb4f9a55fd admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:26:51.718444 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1301/2574] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Sep 23 15:26:51 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 23 15:26:51.722756 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2bd51a-4088-4608-abb9-d5664db60ce5 admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:26:51.734742 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2bd51a-4088-4608-abb9-d5664db60ce5 admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:26:51.750552 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535f909d-cef5-4930-9a5a-2a81ad817f12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:26:51.752763 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535f909d-cef5-4930-9a5a-2a81ad817f12 admin admin] REQUEST_METHOD: `PUT` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:51.755194 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535f909d-cef5-4930-9a5a-2a81ad817f12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:51.755659 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535f909d-cef5-4930-9a5a-2a81ad817f12 admin admin] PATH_INFO: `/v3/projects/4c3bcd0ef2ef401984ac19c1332b95d6/users/2a4c78e36574422ab8695301fc7dadcd/roles/af257be2dee54d6b942f58650196582c` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:51.787125 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535f909d-cef5-4930-9a5a-2a81ad817f12 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c3bcd0ef2ef401984ac19c1332b95d6, user_id=2a4c78e36574422ab8695301fc7dadcd, role_id=af257be2dee54d6b942f58650196582c)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:26:51.790552 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535f909d-cef5-4930-9a5a-2a81ad817f12 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:26:51.940762 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1274/2575] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Sep 23 15:26:51 2022] PUT /identity/v3/projects/4c3bcd0ef2ef401984ac19c1332b95d6/users/2a4c78e36574422ab8695301fc7dadcd/roles/af257be2dee54d6b942f58650196582c => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:26:51.952463 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585933ad-5145-463c-bd42-a85051362b36 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:51.952953 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585933ad-5145-463c-bd42-a85051362b36 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:51.953446 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585933ad-5145-463c-bd42-a85051362b36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:52.000261 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-585933ad-5145-463c-bd42-a85051362b36 None None] MFA Rules not processed for user `2a4c78e36574422ab8695301fc7dadcd`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:26:52.001511 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-585933ad-5145-463c-bd42-a85051362b36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:26:52.087560 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1302/2576] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:26:51 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:26:52.093675 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535f909d-cef5-4930-9a5a-2a81ad817f12 admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:26:52.180214 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535f909d-cef5-4930-9a5a-2a81ad817f12 admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:26:52.186283 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8bd4e3-c544-4156-beb9-23fec4c7a0bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:26:52.186830 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8bd4e3-c544-4156-beb9-23fec4c7a0bd admin admin] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:52.187112 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8bd4e3-c544-4156-beb9-23fec4c7a0bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:52.187381 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8bd4e3-c544-4156-beb9-23fec4c7a0bd admin admin] PATH_INFO: `/v3/projects` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:52.188046 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8bd4e3-c544-4156-beb9-23fec4c7a0bd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:26:52.190273 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8bd4e3-c544-4156-beb9-23fec4c7a0bd admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:26:52.222822 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:26:52.397387 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1275/2577] 10.222.0.55 () {60 vars in 1153 bytes} [Fri Sep 23 15:26:52 2022] POST /identity/v3/projects => generated 393 bytes in 305 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:26:52.401286 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585933ad-5145-463c-bd42-a85051362b36 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:26:52.412313 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585933ad-5145-463c-bd42-a85051362b36 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:26:52.482789 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101b521e-c6f5-4e53-aeb8-6badd7df8b0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:26:52.483611 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101b521e-c6f5-4e53-aeb8-6badd7df8b0b admin admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:52.484069 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101b521e-c6f5-4e53-aeb8-6badd7df8b0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:52.484350 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101b521e-c6f5-4e53-aeb8-6badd7df8b0b admin admin] PATH_INFO: `/v3/users` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:52.484989 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101b521e-c6f5-4e53-aeb8-6badd7df8b0b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:26:52.487056 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101b521e-c6f5-4e53-aeb8-6badd7df8b0b admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:26:52.675228 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1303/2578] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Sep 23 15:26:52 2022] POST /identity/v3/users => generated 303 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:26:52.676823 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8bd4e3-c544-4156-beb9-23fec4c7a0bd admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:26:52.693710 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8bd4e3-c544-4156-beb9-23fec4c7a0bd admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:26:52.729052 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933a1ec3-bb06-438a-9e0a-40eeae731642 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:26:52.729595 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933a1ec3-bb06-438a-9e0a-40eeae731642 admin admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:52.729889 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933a1ec3-bb06-438a-9e0a-40eeae731642 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:52.730164 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933a1ec3-bb06-438a-9e0a-40eeae731642 admin admin] PATH_INFO: `/v3/roles` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:52.730873 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933a1ec3-bb06-438a-9e0a-40eeae731642 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:26:52.733276 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933a1ec3-bb06-438a-9e0a-40eeae731642 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:26:52.754811 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1276/2579] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Sep 23 15:26:52 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:26:52.759373 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101b521e-c6f5-4e53-aeb8-6badd7df8b0b admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:26:52.770906 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101b521e-c6f5-4e53-aeb8-6badd7df8b0b admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:26:52.776097 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b58aa32-21f1-4d8b-b499-e4eebf08ee19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:26:52.776783 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b58aa32-21f1-4d8b-b499-e4eebf08ee19 admin admin] REQUEST_METHOD: `PUT` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:52.777065 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b58aa32-21f1-4d8b-b499-e4eebf08ee19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:52.777346 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b58aa32-21f1-4d8b-b499-e4eebf08ee19 admin admin] PATH_INFO: `/v3/projects/ecb14d720b7542abb1bda678afca7701/users/6d7ea100667a496085b4d63fcfb4a7c5/roles/8e2919e79df040e8926f21607818b58c` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:52.798753 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b58aa32-21f1-4d8b-b499-e4eebf08ee19 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ecb14d720b7542abb1bda678afca7701, user_id=6d7ea100667a496085b4d63fcfb4a7c5, role_id=8e2919e79df040e8926f21607818b58c)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:26:52.801681 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b58aa32-21f1-4d8b-b499-e4eebf08ee19 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:26:52.888400 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1304/2580] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Sep 23 15:26:52 2022] PUT /identity/v3/projects/ecb14d720b7542abb1bda678afca7701/users/6d7ea100667a496085b4d63fcfb4a7c5/roles/8e2919e79df040e8926f21607818b58c => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:26:52.888516 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933a1ec3-bb06-438a-9e0a-40eeae731642 admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:26:52.905107 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933a1ec3-bb06-438a-9e0a-40eeae731642 admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:26:53.010386 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63592f09-414f-4389-8b24-29a7dea25629 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:26:53.011302 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63592f09-414f-4389-8b24-29a7dea25629 admin admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:53.011855 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63592f09-414f-4389-8b24-29a7dea25629 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:53.012428 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63592f09-414f-4389-8b24-29a7dea25629 admin admin] PATH_INFO: `/v3/roles` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:53.017813 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63592f09-414f-4389-8b24-29a7dea25629 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:26:53.019707 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63592f09-414f-4389-8b24-29a7dea25629 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:26:53.042018 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1277/2581] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Sep 23 15:26:52 2022] GET /identity/v3/roles => generated 1420 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:26:53.046912 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b58aa32-21f1-4d8b-b499-e4eebf08ee19 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:26:53.053860 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:26:53.065190 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b58aa32-21f1-4d8b-b499-e4eebf08ee19 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:26:53.090759 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a93319-e8ab-441f-b798-828522afedb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:26:53.092153 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a93319-e8ab-441f-b798-828522afedb3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:53.095792 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a93319-e8ab-441f-b798-828522afedb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:53.095792 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a93319-e8ab-441f-b798-828522afedb3 admin admin] PATH_INFO: `/v3/projects/ecb14d720b7542abb1bda678afca7701/users/6d7ea100667a496085b4d63fcfb4a7c5/roles/8e2919e79df040e8926f21607818b58c` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:53.095792 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a93319-e8ab-441f-b798-828522afedb3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ecb14d720b7542abb1bda678afca7701, user_id=6d7ea100667a496085b4d63fcfb4a7c5, role_id=8e2919e79df040e8926f21607818b58c)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:26:53.099481 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a93319-e8ab-441f-b798-828522afedb3 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:26:53.112454 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1305/2582] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Sep 23 15:26:53 2022] PUT /identity/v3/projects/ecb14d720b7542abb1bda678afca7701/users/6d7ea100667a496085b4d63fcfb4a7c5/roles/8e2919e79df040e8926f21607818b58c => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:26:53.126762 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c95e14e-f94e-46ba-b328-678904b36bbe None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:53.127054 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c95e14e-f94e-46ba-b328-678904b36bbe None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:53.127511 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c95e14e-f94e-46ba-b328-678904b36bbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:53.182410 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-6c95e14e-f94e-46ba-b328-678904b36bbe None None] MFA Rules not processed for user `6d7ea100667a496085b4d63fcfb4a7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:26:53.183939 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-6c95e14e-f94e-46ba-b328-678904b36bbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:26:53.281094 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1278/2583] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:26:53 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:26:53.357390 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a93319-e8ab-441f-b798-828522afedb3 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:26:53.359842 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-73a93319-e8ab-441f-b798-828522afedb3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:26:53.448894 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a93319-e8ab-441f-b798-828522afedb3 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:26:53.461073 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391fd4e6-29f6-4e79-bfcc-f903ae388912 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feca6278be904ef8b0dc580a67e477f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:26:53.461738 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391fd4e6-29f6-4e79-bfcc-f903ae388912 service cinder] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:53.462023 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391fd4e6-29f6-4e79-bfcc-f903ae388912 service cinder] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:53.462302 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391fd4e6-29f6-4e79-bfcc-f903ae388912 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:53.473679 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391fd4e6-29f6-4e79-bfcc-f903ae388912 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:26:53.476160 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391fd4e6-29f6-4e79-bfcc-f903ae388912 service cinder] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:26:53.488339 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1306/2584] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:26:53 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:26:53.553339 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c95e14e-f94e-46ba-b328-678904b36bbe None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:26:53.564049 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c95e14e-f94e-46ba-b328-678904b36bbe None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:26:53.568059 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ab6a78-d2c4-4b9d-9f06-78796d699173 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feca6278be904ef8b0dc580a67e477f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:26:53.579745 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ab6a78-d2c4-4b9d-9f06-78796d699173 service cinder] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:53.581271 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ab6a78-d2c4-4b9d-9f06-78796d699173 service cinder] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:53.581271 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ab6a78-d2c4-4b9d-9f06-78796d699173 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:53.665953 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ab6a78-d2c4-4b9d-9f06-78796d699173 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:26:53.672975 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ab6a78-d2c4-4b9d-9f06-78796d699173 service cinder] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:26:53.678200 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1279/2585] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:26:53 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:26:59.650978 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c406c7-8ad8-442f-9435-b9da44211ad7 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:59.651679 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c406c7-8ad8-442f-9435-b9da44211ad7 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:59.651679 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c406c7-8ad8-442f-9435-b9da44211ad7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:59.673016 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-f6c406c7-8ad8-442f-9435-b9da44211ad7 None None] MFA Rules not processed for user `68206a6b4f244d45bf7bf1504396e430`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:26:59.674244 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-f6c406c7-8ad8-442f-9435-b9da44211ad7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:26:59.706190 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ab6a78-d2c4-4b9d-9f06-78796d699173 service cinder] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:26:59.711434 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:26:59.741431 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1307/2586] 10.222.0.55 () {58 vars in 1009 bytes} [Fri Sep 23 15:26:59 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:26:59.754713 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c406c7-8ad8-442f-9435-b9da44211ad7 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:26:59.756126 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-f6c406c7-8ad8-442f-9435-b9da44211ad7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:26:59.786328 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ab6a78-d2c4-4b9d-9f06-78796d699173 service cinder] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:26:59.791570 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf0ab81-e4dc-4a45-a871-eee93a901e8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:26:59.792239 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf0ab81-e4dc-4a45-a871-eee93a901e8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:59.793152 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf0ab81-e4dc-4a45-a871-eee93a901e8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:59.793152 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf0ab81-e4dc-4a45-a871-eee93a901e8e admin admin] PATH_INFO: `/v3/users/2a4c78e36574422ab8695301fc7dadcd` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:59.794876 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf0ab81-e4dc-4a45-a871-eee93a901e8e admin admin] RBAC: Authorizing `identity:delete_user(user_id=2a4c78e36574422ab8695301fc7dadcd)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:26:59.796923 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf0ab81-e4dc-4a45-a871-eee93a901e8e admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:26:59.847528 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c406c7-8ad8-442f-9435-b9da44211ad7 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:26:59.852837 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb06cc0d-ee5c-4977-86ca-bfd5e788687b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2228c0b49a0249bbaf524a8c4e9e108b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:26:59.853540 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb06cc0d-ee5c-4977-86ca-bfd5e788687b service swift] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:59.853822 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb06cc0d-ee5c-4977-86ca-bfd5e788687b service swift] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:59.854097 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb06cc0d-ee5c-4977-86ca-bfd5e788687b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:59.868263 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb06cc0d-ee5c-4977-86ca-bfd5e788687b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:26:59.868263 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb06cc0d-ee5c-4977-86ca-bfd5e788687b service swift] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:26:59.876926 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1308/2587] 10.222.0.55 () {60 vars in 1384 bytes} [Fri Sep 23 15:26:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 23 15:27:00.033187 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-fbf0ab81-e4dc-4a45-a871-eee93a901e8e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2a4c78e36574422ab8695301fc7dadcd'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:27:00.050930 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-fbf0ab81-e4dc-4a45-a871-eee93a901e8e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2a4c78e36574422ab8695301fc7dadcd'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:27:00.058379 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-fbf0ab81-e4dc-4a45-a871-eee93a901e8e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2a4c78e36574422ab8695301fc7dadcd'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:27:00.180977 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-fbf0ab81-e4dc-4a45-a871-eee93a901e8e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2a4c78e36574422ab8695301fc7dadcd'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:27:00.182995 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-fbf0ab81-e4dc-4a45-a871-eee93a901e8e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2a4c78e36574422ab8695301fc7dadcd'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:27:00.194509 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1280/2588] 10.222.0.55 () {58 vars in 1224 bytes} [Fri Sep 23 15:26:59 2022] DELETE /identity/v3/users/2a4c78e36574422ab8695301fc7dadcd => generated 0 bytes in 486 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:27:00.196442 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc544a39-d924-444f-9ee3-2607de073ac8 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:27:00.197485 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc544a39-d924-444f-9ee3-2607de073ac8 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:27:00.197926 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc544a39-d924-444f-9ee3-2607de073ac8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:27:00.242624 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-cc544a39-d924-444f-9ee3-2607de073ac8 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:27:00.244088 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-cc544a39-d924-444f-9ee3-2607de073ac8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:27:00.281673 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:27:00.328883 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1309/2589] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:27:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:27:00.338950 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf0ab81-e4dc-4a45-a871-eee93a901e8e admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:27:00.340528 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-fbf0ab81-e4dc-4a45-a871-eee93a901e8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:27:00.426709 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf0ab81-e4dc-4a45-a871-eee93a901e8e admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:27:00.431080 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a832c3d-6c97-4305-b2e2-64119a3eb4fc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd259178570864ffeb936ee10dfec6b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:27:00.431882 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a832c3d-6c97-4305-b2e2-64119a3eb4fc service neutron] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:27:00.432216 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a832c3d-6c97-4305-b2e2-64119a3eb4fc service neutron] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:27:00.432758 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a832c3d-6c97-4305-b2e2-64119a3eb4fc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:27:00.443106 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a832c3d-6c97-4305-b2e2-64119a3eb4fc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:27:00.444631 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a832c3d-6c97-4305-b2e2-64119a3eb4fc service neutron] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:27:00.456641 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1281/2590] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:27:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:27:01.008051 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc544a39-d924-444f-9ee3-2607de073ac8 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:27:01.009830 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-cc544a39-d924-444f-9ee3-2607de073ac8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:27:01.030430 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc544a39-d924-444f-9ee3-2607de073ac8 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:27:01.030430 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bee83cb-4ea9-491f-aa0a-8b8660a09bf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:27:01.030430 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bee83cb-4ea9-491f-aa0a-8b8660a09bf7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:27:01.033281 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bee83cb-4ea9-491f-aa0a-8b8660a09bf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:27:01.033281 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bee83cb-4ea9-491f-aa0a-8b8660a09bf7 admin admin] PATH_INFO: `/v3/projects/4c3bcd0ef2ef401984ac19c1332b95d6` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:27:01.033281 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bee83cb-4ea9-491f-aa0a-8b8660a09bf7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4c3bcd0ef2ef401984ac19c1332b95d6)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:27:01.034269 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bee83cb-4ea9-491f-aa0a-8b8660a09bf7 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:27:01.227422 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-2bee83cb-4ea9-491f-aa0a-8b8660a09bf7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4c3bcd0ef2ef401984ac19c1332b95d6', 'request_id': 'req-2bee83cb-4ea9-491f-aa0a-8b8660a09bf7'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:27:01.234532 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-2bee83cb-4ea9-491f-aa0a-8b8660a09bf7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4c3bcd0ef2ef401984ac19c1332b95d6', 'request_id': 'req-2bee83cb-4ea9-491f-aa0a-8b8660a09bf7'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:27:01.240647 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-2bee83cb-4ea9-491f-aa0a-8b8660a09bf7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4c3bcd0ef2ef401984ac19c1332b95d6', 'request_id': 'req-2bee83cb-4ea9-491f-aa0a-8b8660a09bf7'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:27:01.359785 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-2bee83cb-4ea9-491f-aa0a-8b8660a09bf7 admin admin] The token cache is being invalidate because project 4c3bcd0ef2ef401984ac19c1332b95d6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81994) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 15:27:01.360407 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-2bee83cb-4ea9-491f-aa0a-8b8660a09bf7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:27:01.361173 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-2bee83cb-4ea9-491f-aa0a-8b8660a09bf7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:27:01.363145 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1310/2591] 10.222.0.55 () {58 vars in 1233 bytes} [Fri Sep 23 15:27:01 2022] DELETE /identity/v3/projects/4c3bcd0ef2ef401984ac19c1332b95d6 => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:27:01.367118 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a832c3d-6c97-4305-b2e2-64119a3eb4fc service neutron] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:27:01.384984 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a832c3d-6c97-4305-b2e2-64119a3eb4fc service neutron] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:27:01.431351 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:27:01.503510 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d6979a-a268-457f-b3e6-045e8b641295 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:27:01.504455 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d6979a-a268-457f-b3e6-045e8b641295 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:27:01.504905 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d6979a-a268-457f-b3e6-045e8b641295 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:27:01.505330 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d6979a-a268-457f-b3e6-045e8b641295 admin admin] PATH_INFO: `/v3/users/6d7ea100667a496085b4d63fcfb4a7c5` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:27:01.506661 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d6979a-a268-457f-b3e6-045e8b641295 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6d7ea100667a496085b4d63fcfb4a7c5)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:27:01.508625 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d6979a-a268-457f-b3e6-045e8b641295 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:27:01.699912 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-f9d6979a-a268-457f-b3e6-045e8b641295 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6d7ea100667a496085b4d63fcfb4a7c5'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:27:01.723891 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-f9d6979a-a268-457f-b3e6-045e8b641295 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6d7ea100667a496085b4d63fcfb4a7c5'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:27:01.735822 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-f9d6979a-a268-457f-b3e6-045e8b641295 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6d7ea100667a496085b4d63fcfb4a7c5'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:27:01.828542 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-f9d6979a-a268-457f-b3e6-045e8b641295 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6d7ea100667a496085b4d63fcfb4a7c5'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:27:01.829158 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-f9d6979a-a268-457f-b3e6-045e8b641295 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6d7ea100667a496085b4d63fcfb4a7c5'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:27:01.836977 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1282/2592] 10.222.0.55 () {58 vars in 1224 bytes} [Fri Sep 23 15:27:01 2022] DELETE /identity/v3/users/6d7ea100667a496085b4d63fcfb4a7c5 => generated 0 bytes in 470 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:27:02.319374 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bee83cb-4ea9-491f-aa0a-8b8660a09bf7 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:27:02.321798 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-2bee83cb-4ea9-491f-aa0a-8b8660a09bf7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:27:02.336579 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bee83cb-4ea9-491f-aa0a-8b8660a09bf7 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:27:02.422547 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b6d875-1f40-43ca-88c8-7270c333631a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:27:02.423749 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b6d875-1f40-43ca-88c8-7270c333631a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:27:02.424261 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b6d875-1f40-43ca-88c8-7270c333631a admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:27:02.424711 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b6d875-1f40-43ca-88c8-7270c333631a admin admin] PATH_INFO: `/v3/projects/ecb14d720b7542abb1bda678afca7701` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:27:02.426226 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b6d875-1f40-43ca-88c8-7270c333631a admin admin] RBAC: Authorizing `identity:delete_project(project_id=ecb14d720b7542abb1bda678afca7701)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:27:02.430047 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b6d875-1f40-43ca-88c8-7270c333631a admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:27:02.556847 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-88b6d875-1f40-43ca-88c8-7270c333631a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ecb14d720b7542abb1bda678afca7701', 'request_id': 'req-88b6d875-1f40-43ca-88c8-7270c333631a'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:27:02.561167 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:27:02.566249 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-88b6d875-1f40-43ca-88c8-7270c333631a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ecb14d720b7542abb1bda678afca7701', 'request_id': 'req-88b6d875-1f40-43ca-88c8-7270c333631a'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:27:02.573151 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-88b6d875-1f40-43ca-88c8-7270c333631a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ecb14d720b7542abb1bda678afca7701', 'request_id': 'req-88b6d875-1f40-43ca-88c8-7270c333631a'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:27:02.641849 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-88b6d875-1f40-43ca-88c8-7270c333631a admin admin] The token cache is being invalidate because project ecb14d720b7542abb1bda678afca7701 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81994) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 15:27:02.642348 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-88b6d875-1f40-43ca-88c8-7270c333631a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:27:02.643551 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-88b6d875-1f40-43ca-88c8-7270c333631a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:27:02.645796 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1311/2593] 10.222.0.55 () {58 vars in 1233 bytes} [Fri Sep 23 15:27:02 2022] DELETE /identity/v3/projects/ecb14d720b7542abb1bda678afca7701 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:27:02.650465 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0051551-ce05-41b2-89fd-6f451ed4e9f8 None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:27:02.651316 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0051551-ce05-41b2-89fd-6f451ed4e9f8 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:27:02.651780 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0051551-ce05-41b2-89fd-6f451ed4e9f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:27:02.687440 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-e0051551-ce05-41b2-89fd-6f451ed4e9f8 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:27:02.689400 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-e0051551-ce05-41b2-89fd-6f451ed4e9f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:27:02.787953 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1283/2594] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:27:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 15:27:02.797085 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951c850e-3aba-4060-8a8a-08e213ce167f None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:27:02.797590 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951c850e-3aba-4060-8a8a-08e213ce167f None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:27:02.797867 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951c850e-3aba-4060-8a8a-08e213ce167f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:27:02.822098 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-951c850e-3aba-4060-8a8a-08e213ce167f None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:27:02.823314 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-951c850e-3aba-4060-8a8a-08e213ce167f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:27:02.840095 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1312/2595] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:27:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:27:02.843264 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0051551-ce05-41b2-89fd-6f451ed4e9f8 None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:27:02.854470 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0051551-ce05-41b2-89fd-6f451ed4e9f8 None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:27:02.860393 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e4a320-568a-489e-ba2c-6b381db71627 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:27:02.861097 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e4a320-568a-489e-ba2c-6b381db71627 admin admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:27:02.861371 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e4a320-568a-489e-ba2c-6b381db71627 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:27:02.861673 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e4a320-568a-489e-ba2c-6b381db71627 admin admin] PATH_INFO: `/v3/domains` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:27:02.862319 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e4a320-568a-489e-ba2c-6b381db71627 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:27:02.862627 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e4a320-568a-489e-ba2c-6b381db71627 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:27:02.864497 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e4a320-568a-489e-ba2c-6b381db71627 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:27:02.884027 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1284/2596] 10.222.0.55 () {58 vars in 1153 bytes} [Fri Sep 23 15:27:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:27:02.887045 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951c850e-3aba-4060-8a8a-08e213ce167f None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:27:02.900377 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951c850e-3aba-4060-8a8a-08e213ce167f None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:27:02.905878 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45479fe1-9b06-494e-8f84-06a533395049 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:27:02.906499 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45479fe1-9b06-494e-8f84-06a533395049 admin admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:27:02.906779 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45479fe1-9b06-494e-8f84-06a533395049 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:27:02.907050 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45479fe1-9b06-494e-8f84-06a533395049 admin admin] PATH_INFO: `/v3/projects` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:27:02.907748 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45479fe1-9b06-494e-8f84-06a533395049 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:27:02.909707 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45479fe1-9b06-494e-8f84-06a533395049 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:27:03.013024 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1313/2597] 10.222.0.55 () {60 vars in 1153 bytes} [Fri Sep 23 15:27:02 2022] POST /identity/v3/projects => generated 369 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:27:03.030457 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e4a320-568a-489e-ba2c-6b381db71627 admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:27:03.057916 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e4a320-568a-489e-ba2c-6b381db71627 admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:27:03.109439 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:27:03.155669 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b808b3de-0064-430f-9899-0eda61ba4c54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:27:03.156412 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b808b3de-0064-430f-9899-0eda61ba4c54 admin admin] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:27:03.156836 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b808b3de-0064-430f-9899-0eda61ba4c54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:27:03.157307 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b808b3de-0064-430f-9899-0eda61ba4c54 admin admin] PATH_INFO: `/v3/users` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:27:03.158020 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b808b3de-0064-430f-9899-0eda61ba4c54 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:27:03.160153 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b808b3de-0064-430f-9899-0eda61ba4c54 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:27:03.274970 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1285/2598] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Sep 23 15:27:03 2022] POST /identity/v3/users => generated 285 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:27:03.280358 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45479fe1-9b06-494e-8f84-06a533395049 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:27:03.291700 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45479fe1-9b06-494e-8f84-06a533395049 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:27:03.298631 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126383e6-da1d-4c8d-9247-79be8b82e8bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:27:03.299377 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126383e6-da1d-4c8d-9247-79be8b82e8bd admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:27:03.299736 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126383e6-da1d-4c8d-9247-79be8b82e8bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:27:03.300116 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126383e6-da1d-4c8d-9247-79be8b82e8bd admin admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:27:03.300937 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126383e6-da1d-4c8d-9247-79be8b82e8bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:27:03.302693 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126383e6-da1d-4c8d-9247-79be8b82e8bd admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:27:03.313794 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1314/2599] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Sep 23 15:27:03 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 23 15:27:03.318222 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b808b3de-0064-430f-9899-0eda61ba4c54 admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:27:03.331928 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b808b3de-0064-430f-9899-0eda61ba4c54 admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:27:03.336735 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24c5e6c-9f92-4358-adff-7ba382d5157e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:27:03.337424 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24c5e6c-9f92-4358-adff-7ba382d5157e admin admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:27:03.337816 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24c5e6c-9f92-4358-adff-7ba382d5157e admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:27:03.338140 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24c5e6c-9f92-4358-adff-7ba382d5157e admin admin] PATH_INFO: `/v3/roles` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:27:03.338874 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24c5e6c-9f92-4358-adff-7ba382d5157e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:27:03.340692 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24c5e6c-9f92-4358-adff-7ba382d5157e admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:27:03.355244 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1286/2600] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Sep 23 15:27:03 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:27:03.360628 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126383e6-da1d-4c8d-9247-79be8b82e8bd admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:27:03.371508 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126383e6-da1d-4c8d-9247-79be8b82e8bd admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:27:03.379845 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96293a6-d419-4176-830e-5ebdb19b04d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:27:03.380620 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96293a6-d419-4176-830e-5ebdb19b04d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:27:03.380965 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96293a6-d419-4176-830e-5ebdb19b04d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:27:03.381246 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96293a6-d419-4176-830e-5ebdb19b04d9 admin admin] PATH_INFO: `/v3/projects/6bfe7f53f2ae4659a658c188ed864ff5/users/59bc91d3d9634dad8bf3e68e25c159b5/roles/af257be2dee54d6b942f58650196582c` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:27:03.398222 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96293a6-d419-4176-830e-5ebdb19b04d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6bfe7f53f2ae4659a658c188ed864ff5, user_id=59bc91d3d9634dad8bf3e68e25c159b5, role_id=af257be2dee54d6b942f58650196582c)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:27:03.401473 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96293a6-d419-4176-830e-5ebdb19b04d9 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:27:03.510496 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1315/2601] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Sep 23 15:27:03 2022] PUT /identity/v3/projects/6bfe7f53f2ae4659a658c188ed864ff5/users/59bc91d3d9634dad8bf3e68e25c159b5/roles/af257be2dee54d6b942f58650196582c => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 15:27:03.516917 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575d1dc0-0ba2-4945-aec9-c25c50d4e89e None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:27:03.517320 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575d1dc0-0ba2-4945-aec9-c25c50d4e89e None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:27:03.517739 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575d1dc0-0ba2-4945-aec9-c25c50d4e89e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:27:03.541974 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-575d1dc0-0ba2-4945-aec9-c25c50d4e89e None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:27:03.543210 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-575d1dc0-0ba2-4945-aec9-c25c50d4e89e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:27:03.614549 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1287/2602] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:27:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:27:03.628677 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96293a6-d419-4176-830e-5ebdb19b04d9 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:27:03.631149 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-f96293a6-d419-4176-830e-5ebdb19b04d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:27:03.708830 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96293a6-d419-4176-830e-5ebdb19b04d9 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:27:03.715156 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5eb9b4e-9f1c-4a7c-af13-7b275a9638ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd259178570864ffeb936ee10dfec6b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:27:03.718324 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5eb9b4e-9f1c-4a7c-af13-7b275a9638ef service neutron] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:27:03.718535 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5eb9b4e-9f1c-4a7c-af13-7b275a9638ef service neutron] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:27:03.719557 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5eb9b4e-9f1c-4a7c-af13-7b275a9638ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:27:03.743899 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5eb9b4e-9f1c-4a7c-af13-7b275a9638ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:27:03.743899 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5eb9b4e-9f1c-4a7c-af13-7b275a9638ef service neutron] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:27:03.773780 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1316/2603] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:27:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:27:05.474554 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c20937e-4c60-4ab3-b6b1-edbe572e6fc0 None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:27:05.475490 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c20937e-4c60-4ab3-b6b1-edbe572e6fc0 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:27:05.475948 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c20937e-4c60-4ab3-b6b1-edbe572e6fc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:27:05.544292 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-7c20937e-4c60-4ab3-b6b1-edbe572e6fc0 None None] MFA Rules not processed for user `59bc91d3d9634dad8bf3e68e25c159b5`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:27:05.546150 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-7c20937e-4c60-4ab3-b6b1-edbe572e6fc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:27:05.673527 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1288/2604] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:27:05 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:27:05.677966 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5eb9b4e-9f1c-4a7c-af13-7b275a9638ef service neutron] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:27:05.689369 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5eb9b4e-9f1c-4a7c-af13-7b275a9638ef service neutron] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:27:05.698961 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abb8d6d-a057-4bca-8675-2ee57218eec5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:27:05.699668 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abb8d6d-a057-4bca-8675-2ee57218eec5 admin admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:27:05.700258 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abb8d6d-a057-4bca-8675-2ee57218eec5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:27:05.700258 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abb8d6d-a057-4bca-8675-2ee57218eec5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:27:05.701105 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abb8d6d-a057-4bca-8675-2ee57218eec5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:27:05.703439 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abb8d6d-a057-4bca-8675-2ee57218eec5 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:27:05.833208 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1317/2605] 10.222.0.55 () {60 vars in 1153 bytes} [Fri Sep 23 15:27:05 2022] POST /identity/v3/projects => generated 373 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:27:05.837419 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c20937e-4c60-4ab3-b6b1-edbe572e6fc0 None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:27:05.849635 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c20937e-4c60-4ab3-b6b1-edbe572e6fc0 None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:27:05.890392 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:27:05.911487 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cd9ba5-d9b9-4e53-9db3-ffc7ebe293bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:27:05.912330 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cd9ba5-d9b9-4e53-9db3-ffc7ebe293bd admin admin] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:27:05.912619 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cd9ba5-d9b9-4e53-9db3-ffc7ebe293bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:27:05.912894 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cd9ba5-d9b9-4e53-9db3-ffc7ebe293bd admin admin] PATH_INFO: `/v3/users` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:27:05.913563 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03cd9ba5-d9b9-4e53-9db3-ffc7ebe293bd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:27:05.915504 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03cd9ba5-d9b9-4e53-9db3-ffc7ebe293bd admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:27:06.015618 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1289/2606] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Sep 23 15:27:05 2022] POST /identity/v3/users => generated 293 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:27:06.026418 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abb8d6d-a057-4bca-8675-2ee57218eec5 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:27:06.031541 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abb8d6d-a057-4bca-8675-2ee57218eec5 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:27:06.037093 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f045db72-e67b-43e0-928a-67db404381f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:27:06.037233 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f045db72-e67b-43e0-928a-67db404381f9 admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:27:06.037529 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f045db72-e67b-43e0-928a-67db404381f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:27:06.037813 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f045db72-e67b-43e0-928a-67db404381f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:27:06.038509 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f045db72-e67b-43e0-928a-67db404381f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:27:06.040428 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f045db72-e67b-43e0-928a-67db404381f9 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:27:06.059131 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1318/2607] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Sep 23 15:27:06 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:27:06.061783 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cd9ba5-d9b9-4e53-9db3-ffc7ebe293bd admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:27:06.073091 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cd9ba5-d9b9-4e53-9db3-ffc7ebe293bd admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:27:06.086562 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3049a455-1dd7-45c2-90af-3bd4daeea0c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:27:06.087391 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3049a455-1dd7-45c2-90af-3bd4daeea0c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:27:06.087900 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3049a455-1dd7-45c2-90af-3bd4daeea0c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:27:06.088250 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3049a455-1dd7-45c2-90af-3bd4daeea0c1 admin admin] PATH_INFO: `/v3/projects/cb6172fe14214b55b23c6751210e872c/users/2314d15f11e84200bccb3481300e4a72/roles/8e2919e79df040e8926f21607818b58c` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:27:06.109098 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3049a455-1dd7-45c2-90af-3bd4daeea0c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb6172fe14214b55b23c6751210e872c, user_id=2314d15f11e84200bccb3481300e4a72, role_id=8e2919e79df040e8926f21607818b58c)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:27:06.112680 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3049a455-1dd7-45c2-90af-3bd4daeea0c1 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:27:06.339368 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1290/2608] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Sep 23 15:27:06 2022] PUT /identity/v3/projects/cb6172fe14214b55b23c6751210e872c/users/2314d15f11e84200bccb3481300e4a72/roles/8e2919e79df040e8926f21607818b58c => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:27:06.347763 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f045db72-e67b-43e0-928a-67db404381f9 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:27:06.361211 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f045db72-e67b-43e0-928a-67db404381f9 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:27:06.424314 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:27:06.438134 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e3ee1a-5db2-4b55-82ef-043daa157995 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:27:06.438839 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e3ee1a-5db2-4b55-82ef-043daa157995 admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:27:06.439240 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e3ee1a-5db2-4b55-82ef-043daa157995 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:27:06.439471 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e3ee1a-5db2-4b55-82ef-043daa157995 admin admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:27:06.440167 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e3ee1a-5db2-4b55-82ef-043daa157995 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:27:06.442593 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e3ee1a-5db2-4b55-82ef-043daa157995 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:27:06.454360 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1319/2609] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Sep 23 15:27:06 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:27:06.458746 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3049a455-1dd7-45c2-90af-3bd4daeea0c1 admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:27:06.472405 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3049a455-1dd7-45c2-90af-3bd4daeea0c1 admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:27:06.475014 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd5c2f5-0ab2-424a-925d-40898480a9ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:27:06.475879 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd5c2f5-0ab2-424a-925d-40898480a9ac admin admin] REQUEST_METHOD: `PUT` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:27:06.476117 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd5c2f5-0ab2-424a-925d-40898480a9ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:27:06.476396 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd5c2f5-0ab2-424a-925d-40898480a9ac admin admin] PATH_INFO: `/v3/projects/cb6172fe14214b55b23c6751210e872c/users/2314d15f11e84200bccb3481300e4a72/roles/8e2919e79df040e8926f21607818b58c` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:27:06.478334 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd5c2f5-0ab2-424a-925d-40898480a9ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb6172fe14214b55b23c6751210e872c, user_id=2314d15f11e84200bccb3481300e4a72, role_id=8e2919e79df040e8926f21607818b58c)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:27:06.481017 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd5c2f5-0ab2-424a-925d-40898480a9ac admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:27:06.504136 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1291/2610] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Sep 23 15:27:06 2022] PUT /identity/v3/projects/cb6172fe14214b55b23c6751210e872c/users/2314d15f11e84200bccb3481300e4a72/roles/8e2919e79df040e8926f21607818b58c => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:27:09.117642 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e316c06a-31d9-488b-b6d5-7a980f74259d None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:27:09.118282 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e316c06a-31d9-488b-b6d5-7a980f74259d None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:27:09.118282 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e316c06a-31d9-488b-b6d5-7a980f74259d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:27:09.150106 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-e316c06a-31d9-488b-b6d5-7a980f74259d None None] MFA Rules not processed for user `2314d15f11e84200bccb3481300e4a72`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:27:09.151372 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-e316c06a-31d9-488b-b6d5-7a980f74259d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:27:09.224669 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1320/2611] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:27:09 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:27:09.236514 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd5c2f5-0ab2-424a-925d-40898480a9ac admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:27:09.238344 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-1fd5c2f5-0ab2-424a-925d-40898480a9ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:27:09.308631 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd5c2f5-0ab2-424a-925d-40898480a9ac admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:27:09.313216 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db2e162-8add-4f05-b2da-a30f529171cc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feca6278be904ef8b0dc580a67e477f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:27:09.314110 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db2e162-8add-4f05-b2da-a30f529171cc service cinder] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:27:09.314392 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db2e162-8add-4f05-b2da-a30f529171cc service cinder] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:27:09.314693 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db2e162-8add-4f05-b2da-a30f529171cc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:27:09.376607 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db2e162-8add-4f05-b2da-a30f529171cc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:27:09.378042 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db2e162-8add-4f05-b2da-a30f529171cc service cinder] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:27:09.387027 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1292/2612] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:27:09 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:27:11.398020 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e316c06a-31d9-488b-b6d5-7a980f74259d None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:27:11.408757 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e316c06a-31d9-488b-b6d5-7a980f74259d None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:27:11.412744 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25095f4a-197b-4c80-8f6a-ed7803425e04 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feca6278be904ef8b0dc580a67e477f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:27:11.413393 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25095f4a-197b-4c80-8f6a-ed7803425e04 service cinder] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:27:11.414788 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25095f4a-197b-4c80-8f6a-ed7803425e04 service cinder] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:27:11.414788 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25095f4a-197b-4c80-8f6a-ed7803425e04 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:27:11.424232 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25095f4a-197b-4c80-8f6a-ed7803425e04 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:27:11.425668 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25095f4a-197b-4c80-8f6a-ed7803425e04 service cinder] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:27:11.436053 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1321/2613] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:27:11 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:27:11.758512 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62127e40-d385-4ac7-92f0-d859c002318a None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:27:11.758512 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62127e40-d385-4ac7-92f0-d859c002318a None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:27:11.758512 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62127e40-d385-4ac7-92f0-d859c002318a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:27:11.795066 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-62127e40-d385-4ac7-92f0-d859c002318a None None] MFA Rules not processed for user `1cb120cee6bf4ef4b7da15735f5f77e1`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:27:11.796660 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-62127e40-d385-4ac7-92f0-d859c002318a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:27:11.847095 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:27:11.867532 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1293/2614] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:27:11 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:27:12.232323 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25095f4a-197b-4c80-8f6a-ed7803425e04 service cinder] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:27:12.234339 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-25095f4a-197b-4c80-8f6a-ed7803425e04 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:27:12.294858 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25095f4a-197b-4c80-8f6a-ed7803425e04 service cinder] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:27:12.300400 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1def3d2f-e823-445b-a412-ffff69c011dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd259178570864ffeb936ee10dfec6b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:27:12.301166 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1def3d2f-e823-445b-a412-ffff69c011dc service neutron] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:27:12.301451 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1def3d2f-e823-445b-a412-ffff69c011dc service neutron] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:27:12.301726 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1def3d2f-e823-445b-a412-ffff69c011dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:27:12.303473 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-1def3d2f-e823-445b-a412-ffff69c011dc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:27:12.315856 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1def3d2f-e823-445b-a412-ffff69c011dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:27:12.322713 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1def3d2f-e823-445b-a412-ffff69c011dc service neutron] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:27:12.326277 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1322/2615] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:27:12 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:27:17.032196 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0c7951-b8fa-4ecf-8f4b-7e70c189381f None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:27:17.032731 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0c7951-b8fa-4ecf-8f4b-7e70c189381f None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:27:17.032862 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0c7951-b8fa-4ecf-8f4b-7e70c189381f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:27:17.059498 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-9c0c7951-b8fa-4ecf-8f4b-7e70c189381f None None] MFA Rules not processed for user `68206a6b4f244d45bf7bf1504396e430`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:27:17.059498 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-9c0c7951-b8fa-4ecf-8f4b-7e70c189381f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:27:17.121238 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1294/2616] 10.222.0.55 () {58 vars in 967 bytes} [Fri Sep 23 15:27:17 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:27:17.145196 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1def3d2f-e823-445b-a412-ffff69c011dc service neutron] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:27:17.147106 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-1def3d2f-e823-445b-a412-ffff69c011dc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:27:17.207213 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:27:17.222431 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1def3d2f-e823-445b-a412-ffff69c011dc service neutron] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:27:17.228341 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c06dd6-4a5a-4a3c-b408-91bd82c7db37 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2228c0b49a0249bbaf524a8c4e9e108b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:27:17.229001 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c06dd6-4a5a-4a3c-b408-91bd82c7db37 service swift] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:27:17.229248 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c06dd6-4a5a-4a3c-b408-91bd82c7db37 service swift] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:27:17.229525 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c06dd6-4a5a-4a3c-b408-91bd82c7db37 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:27:17.238032 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c06dd6-4a5a-4a3c-b408-91bd82c7db37 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:27:17.239640 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c06dd6-4a5a-4a3c-b408-91bd82c7db37 service swift] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:27:17.248169 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1323/2617] 10.222.0.55 () {60 vars in 1384 bytes} [Fri Sep 23 15:27:17 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 23 15:27:27.113521 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=29 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:27:27.402953 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=6 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:27:29.917319 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0c7951-b8fa-4ecf-8f4b-7e70c189381f None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:27:29.926929 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0c7951-b8fa-4ecf-8f4b-7e70c189381f None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:27:29.930503 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8e0727-a3e4-459a-ae4f-8426c618ced2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd259178570864ffeb936ee10dfec6b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:27:29.931110 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8e0727-a3e4-459a-ae4f-8426c618ced2 service neutron] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:27:29.931391 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8e0727-a3e4-459a-ae4f-8426c618ced2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:27:29.931666 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8e0727-a3e4-459a-ae4f-8426c618ced2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:27:29.933429 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-1a8e0727-a3e4-459a-ae4f-8426c618ced2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:27:30.001183 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8e0727-a3e4-459a-ae4f-8426c618ced2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:27:30.002659 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8e0727-a3e4-459a-ae4f-8426c618ced2 service neutron] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:27:30.013235 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1295/2618] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:27:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:27:30.601765 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c06dd6-4a5a-4a3c-b408-91bd82c7db37 service swift] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:27:30.612962 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c06dd6-4a5a-4a3c-b408-91bd82c7db37 service swift] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:27:30.620981 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e558a056-066d-40f5-9fd0-4fae95d8ff87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:27:30.621679 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e558a056-066d-40f5-9fd0-4fae95d8ff87 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:27:30.622008 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e558a056-066d-40f5-9fd0-4fae95d8ff87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:27:30.622280 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e558a056-066d-40f5-9fd0-4fae95d8ff87 admin admin] PATH_INFO: `/v3/users/59bc91d3d9634dad8bf3e68e25c159b5` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:27:30.623430 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e558a056-066d-40f5-9fd0-4fae95d8ff87 admin admin] RBAC: Authorizing `identity:delete_user(user_id=59bc91d3d9634dad8bf3e68e25c159b5)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:27:30.625485 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e558a056-066d-40f5-9fd0-4fae95d8ff87 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:27:30.805270 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-e558a056-066d-40f5-9fd0-4fae95d8ff87 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '59bc91d3d9634dad8bf3e68e25c159b5'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:27:30.819626 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-e558a056-066d-40f5-9fd0-4fae95d8ff87 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '59bc91d3d9634dad8bf3e68e25c159b5'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:27:30.839761 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-e558a056-066d-40f5-9fd0-4fae95d8ff87 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '59bc91d3d9634dad8bf3e68e25c159b5'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:27:30.987620 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-e558a056-066d-40f5-9fd0-4fae95d8ff87 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '59bc91d3d9634dad8bf3e68e25c159b5'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:27:30.988191 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-e558a056-066d-40f5-9fd0-4fae95d8ff87 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '59bc91d3d9634dad8bf3e68e25c159b5'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:27:30.995684 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1324/2619] 10.222.0.55 () {58 vars in 1224 bytes} [Fri Sep 23 15:27:30 2022] DELETE /identity/v3/users/59bc91d3d9634dad8bf3e68e25c159b5 => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:27:31.377682 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8e0727-a3e4-459a-ae4f-8426c618ced2 service neutron] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:27:31.391427 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8e0727-a3e4-459a-ae4f-8426c618ced2 service neutron] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:27:31.462546 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10aa4af-9f18-46fe-a220-7dbdd5389b99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:27:31.463602 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10aa4af-9f18-46fe-a220-7dbdd5389b99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:27:31.464072 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10aa4af-9f18-46fe-a220-7dbdd5389b99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:27:31.464531 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10aa4af-9f18-46fe-a220-7dbdd5389b99 admin admin] PATH_INFO: `/v3/projects/6bfe7f53f2ae4659a658c188ed864ff5` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:27:31.465862 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10aa4af-9f18-46fe-a220-7dbdd5389b99 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6bfe7f53f2ae4659a658c188ed864ff5)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:27:31.468216 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10aa4af-9f18-46fe-a220-7dbdd5389b99 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:27:31.591832 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-e10aa4af-9f18-46fe-a220-7dbdd5389b99 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6bfe7f53f2ae4659a658c188ed864ff5', 'request_id': 'req-e10aa4af-9f18-46fe-a220-7dbdd5389b99'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:27:31.601312 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-e10aa4af-9f18-46fe-a220-7dbdd5389b99 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6bfe7f53f2ae4659a658c188ed864ff5', 'request_id': 'req-e10aa4af-9f18-46fe-a220-7dbdd5389b99'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:27:31.724398 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-e10aa4af-9f18-46fe-a220-7dbdd5389b99 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6bfe7f53f2ae4659a658c188ed864ff5', 'request_id': 'req-e10aa4af-9f18-46fe-a220-7dbdd5389b99'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:27:31.737119 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-e10aa4af-9f18-46fe-a220-7dbdd5389b99 admin admin] The token cache is being invalidate because project 6bfe7f53f2ae4659a658c188ed864ff5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81993) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 15:27:31.737727 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-e10aa4af-9f18-46fe-a220-7dbdd5389b99 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:27:31.738956 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-e10aa4af-9f18-46fe-a220-7dbdd5389b99 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:27:31.740735 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1296/2620] 10.222.0.55 () {58 vars in 1233 bytes} [Fri Sep 23 15:27:31 2022] DELETE /identity/v3/projects/6bfe7f53f2ae4659a658c188ed864ff5 => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:27:31.747235 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e558a056-066d-40f5-9fd0-4fae95d8ff87 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:27:31.752784 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-e558a056-066d-40f5-9fd0-4fae95d8ff87 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:27:31.765063 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e558a056-066d-40f5-9fd0-4fae95d8ff87 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:27:31.986126 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc339f6a-a42f-4aa8-87ca-c628eccf5a1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:27:31.987174 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc339f6a-a42f-4aa8-87ca-c628eccf5a1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:27:31.987530 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc339f6a-a42f-4aa8-87ca-c628eccf5a1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:27:31.987875 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc339f6a-a42f-4aa8-87ca-c628eccf5a1a admin admin] PATH_INFO: `/v3/users/2314d15f11e84200bccb3481300e4a72` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:27:31.989597 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc339f6a-a42f-4aa8-87ca-c628eccf5a1a admin admin] RBAC: Authorizing `identity:delete_user(user_id=2314d15f11e84200bccb3481300e4a72)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:27:31.991818 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc339f6a-a42f-4aa8-87ca-c628eccf5a1a admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:27:32.221216 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-fc339f6a-a42f-4aa8-87ca-c628eccf5a1a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2314d15f11e84200bccb3481300e4a72'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:27:32.239765 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-fc339f6a-a42f-4aa8-87ca-c628eccf5a1a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2314d15f11e84200bccb3481300e4a72'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:27:32.249724 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-fc339f6a-a42f-4aa8-87ca-c628eccf5a1a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2314d15f11e84200bccb3481300e4a72'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:27:32.252484 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=80,keystone:DELETE=19,keystone:INSERT=1 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:27:32.337582 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-fc339f6a-a42f-4aa8-87ca-c628eccf5a1a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2314d15f11e84200bccb3481300e4a72'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:27:32.338363 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-fc339f6a-a42f-4aa8-87ca-c628eccf5a1a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2314d15f11e84200bccb3481300e4a72'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:27:32.346700 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1325/2621] 10.222.0.55 () {58 vars in 1224 bytes} [Fri Sep 23 15:27:31 2022] DELETE /identity/v3/users/2314d15f11e84200bccb3481300e4a72 => generated 0 bytes in 601 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:27:32.697775 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10aa4af-9f18-46fe-a220-7dbdd5389b99 admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:27:32.700536 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-e10aa4af-9f18-46fe-a220-7dbdd5389b99 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:27:32.739466 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10aa4af-9f18-46fe-a220-7dbdd5389b99 admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:27:32.765488 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:27:32.808217 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c085ef-d284-4455-817a-263f5414bbf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:27:32.812817 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c085ef-d284-4455-817a-263f5414bbf4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:27:32.812817 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c085ef-d284-4455-817a-263f5414bbf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:27:32.812817 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c085ef-d284-4455-817a-263f5414bbf4 admin admin] PATH_INFO: `/v3/projects/cb6172fe14214b55b23c6751210e872c` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:27:32.812817 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c085ef-d284-4455-817a-263f5414bbf4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb6172fe14214b55b23c6751210e872c)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:27:32.812817 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c085ef-d284-4455-817a-263f5414bbf4 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:27:32.999326 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-37c085ef-d284-4455-817a-263f5414bbf4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb6172fe14214b55b23c6751210e872c', 'request_id': 'req-37c085ef-d284-4455-817a-263f5414bbf4'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:27:33.005844 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-37c085ef-d284-4455-817a-263f5414bbf4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb6172fe14214b55b23c6751210e872c', 'request_id': 'req-37c085ef-d284-4455-817a-263f5414bbf4'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:27:33.130014 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-37c085ef-d284-4455-817a-263f5414bbf4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb6172fe14214b55b23c6751210e872c', 'request_id': 'req-37c085ef-d284-4455-817a-263f5414bbf4'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:27:33.145530 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-37c085ef-d284-4455-817a-263f5414bbf4 admin admin] The token cache is being invalidate because project cb6172fe14214b55b23c6751210e872c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81993) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 15:27:33.151034 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-37c085ef-d284-4455-817a-263f5414bbf4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:27:33.151034 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-37c085ef-d284-4455-817a-263f5414bbf4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:27:33.151034 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1297/2622] 10.222.0.55 () {58 vars in 1233 bytes} [Fri Sep 23 15:27:32 2022] DELETE /identity/v3/projects/cb6172fe14214b55b23c6751210e872c => generated 0 bytes in 451 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:27:33.153272 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c98f093-d18f-429f-8e75-8af789ae221f None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:27:33.153744 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c98f093-d18f-429f-8e75-8af789ae221f None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:27:33.153954 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c98f093-d18f-429f-8e75-8af789ae221f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:27:33.177863 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-0c98f093-d18f-429f-8e75-8af789ae221f None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:27:33.181398 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-0c98f093-d18f-429f-8e75-8af789ae221f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:27:33.328846 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1326/2623] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:27:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:27:33.337347 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40f164f-89b5-4f09-a11c-096c9460ec9d None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:27:33.337579 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40f164f-89b5-4f09-a11c-096c9460ec9d None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:27:33.337826 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40f164f-89b5-4f09-a11c-096c9460ec9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:27:33.368023 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-a40f164f-89b5-4f09-a11c-096c9460ec9d None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:27:33.369559 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-a40f164f-89b5-4f09-a11c-096c9460ec9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:27:33.380144 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1298/2624] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:27:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:27:33.384166 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c98f093-d18f-429f-8e75-8af789ae221f None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:27:33.398377 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c98f093-d18f-429f-8e75-8af789ae221f None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:27:33.405192 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bf41d3-5dfb-4512-83b5-d449bdeb691e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:27:33.405907 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bf41d3-5dfb-4512-83b5-d449bdeb691e admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:27:33.406702 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bf41d3-5dfb-4512-83b5-d449bdeb691e admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:27:33.407053 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bf41d3-5dfb-4512-83b5-d449bdeb691e admin admin] PATH_INFO: `/v3/domains` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:27:33.407782 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bf41d3-5dfb-4512-83b5-d449bdeb691e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:27:33.408185 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bf41d3-5dfb-4512-83b5-d449bdeb691e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:27:33.409853 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bf41d3-5dfb-4512-83b5-d449bdeb691e admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:27:33.429659 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1327/2625] 10.222.0.55 () {58 vars in 1153 bytes} [Fri Sep 23 15:27:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 15:27:33.434051 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40f164f-89b5-4f09-a11c-096c9460ec9d None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:27:33.445280 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40f164f-89b5-4f09-a11c-096c9460ec9d None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:27:33.469914 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d089934e-c114-4ddd-8167-c1e5e7cd8140 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:27:33.470551 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d089934e-c114-4ddd-8167-c1e5e7cd8140 admin admin] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:27:33.470833 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d089934e-c114-4ddd-8167-c1e5e7cd8140 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:27:33.471134 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d089934e-c114-4ddd-8167-c1e5e7cd8140 admin admin] PATH_INFO: `/v3/projects` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:27:33.471759 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d089934e-c114-4ddd-8167-c1e5e7cd8140 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:27:33.473798 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d089934e-c114-4ddd-8167-c1e5e7cd8140 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:27:33.618799 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1299/2626] 10.222.0.55 () {60 vars in 1153 bytes} [Fri Sep 23 15:27:33 2022] POST /identity/v3/projects => generated 385 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:27:33.623012 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bf41d3-5dfb-4512-83b5-d449bdeb691e admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:27:33.639269 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bf41d3-5dfb-4512-83b5-d449bdeb691e admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:27:33.713067 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdcf115-d6af-4eb8-bd58-6f586fcf448a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:27:33.713660 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdcf115-d6af-4eb8-bd58-6f586fcf448a admin admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:27:33.713949 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdcf115-d6af-4eb8-bd58-6f586fcf448a admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:27:33.714226 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdcf115-d6af-4eb8-bd58-6f586fcf448a admin admin] PATH_INFO: `/v3/users` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:27:33.715046 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdcf115-d6af-4eb8-bd58-6f586fcf448a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:27:33.717068 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdcf115-d6af-4eb8-bd58-6f586fcf448a admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:27:33.883020 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1328/2627] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Sep 23 15:27:33 2022] POST /identity/v3/users => generated 293 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:27:33.889527 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d089934e-c114-4ddd-8167-c1e5e7cd8140 admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:27:33.898237 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d089934e-c114-4ddd-8167-c1e5e7cd8140 admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:27:33.904286 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305f34cb-1a53-48bd-9e6c-659bc74d87c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:27:33.904908 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305f34cb-1a53-48bd-9e6c-659bc74d87c0 admin admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:27:33.905178 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305f34cb-1a53-48bd-9e6c-659bc74d87c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:27:33.905453 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305f34cb-1a53-48bd-9e6c-659bc74d87c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:27:33.906098 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305f34cb-1a53-48bd-9e6c-659bc74d87c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:27:33.907700 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305f34cb-1a53-48bd-9e6c-659bc74d87c0 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:27:33.919390 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1300/2628] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Sep 23 15:27:33 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:27:33.923489 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdcf115-d6af-4eb8-bd58-6f586fcf448a admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:27:33.935957 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdcf115-d6af-4eb8-bd58-6f586fcf448a admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:27:33.940828 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafa0592-3795-4a01-8b2d-d9bd0ab29c82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:27:33.941791 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafa0592-3795-4a01-8b2d-d9bd0ab29c82 admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:27:33.942222 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafa0592-3795-4a01-8b2d-d9bd0ab29c82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:27:33.942535 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafa0592-3795-4a01-8b2d-d9bd0ab29c82 admin admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:27:33.943139 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafa0592-3795-4a01-8b2d-d9bd0ab29c82 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:27:33.944800 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafa0592-3795-4a01-8b2d-d9bd0ab29c82 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:27:33.973822 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1329/2629] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Sep 23 15:27:33 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:27:33.976513 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305f34cb-1a53-48bd-9e6c-659bc74d87c0 admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:27:33.993195 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305f34cb-1a53-48bd-9e6c-659bc74d87c0 admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:27:33.998455 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b784edf-af10-4aa2-884e-dd2bfbc5dfe7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:27:33.999947 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b784edf-af10-4aa2-884e-dd2bfbc5dfe7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:27:33.999947 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b784edf-af10-4aa2-884e-dd2bfbc5dfe7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:27:33.999947 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b784edf-af10-4aa2-884e-dd2bfbc5dfe7 admin admin] PATH_INFO: `/v3/projects/aaed7382e3b34d449183114c31315ace/users/833c1526634a402b870129ee3f3f008f/roles/af257be2dee54d6b942f58650196582c` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:27:34.017693 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b784edf-af10-4aa2-884e-dd2bfbc5dfe7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aaed7382e3b34d449183114c31315ace, user_id=833c1526634a402b870129ee3f3f008f, role_id=af257be2dee54d6b942f58650196582c)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:27:34.021459 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b784edf-af10-4aa2-884e-dd2bfbc5dfe7 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:27:34.121073 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1301/2630] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Sep 23 15:27:33 2022] PUT /identity/v3/projects/aaed7382e3b34d449183114c31315ace/users/833c1526634a402b870129ee3f3f008f/roles/af257be2dee54d6b942f58650196582c => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:27:34.131914 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8247427d-d4b5-48cd-8f2c-dd6777d53536 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:27:34.132311 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8247427d-d4b5-48cd-8f2c-dd6777d53536 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:27:34.132539 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8247427d-d4b5-48cd-8f2c-dd6777d53536 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:27:34.161118 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b784edf-af10-4aa2-884e-dd2bfbc5dfe7 admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:27:34.163829 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-4b784edf-af10-4aa2-884e-dd2bfbc5dfe7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:27:34.164990 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-8247427d-d4b5-48cd-8f2c-dd6777d53536 None None] MFA Rules not processed for user `833c1526634a402b870129ee3f3f008f`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:27:34.166795 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-8247427d-d4b5-48cd-8f2c-dd6777d53536 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:27:34.216946 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:INSERT=4,keystone:DELETE=1,keystone:SELECT=95 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:27:34.223665 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:27:34.236580 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b784edf-af10-4aa2-884e-dd2bfbc5dfe7 admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:27:34.239499 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1330/2631] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:27:34 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:27:34.242275 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d65fcf9-1cec-42e3-b0a2-3ac8b80f335f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:27:34.243194 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d65fcf9-1cec-42e3-b0a2-3ac8b80f335f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:27:34.243480 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d65fcf9-1cec-42e3-b0a2-3ac8b80f335f admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:27:34.243756 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d65fcf9-1cec-42e3-b0a2-3ac8b80f335f admin admin] PATH_INFO: `/v3/users/f5f0ecaf38e74c96bc439fde75d93d37` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:27:34.244171 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8247427d-d4b5-48cd-8f2c-dd6777d53536 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:27:34.244975 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d65fcf9-1cec-42e3-b0a2-3ac8b80f335f admin admin] RBAC: Authorizing `identity:delete_user(user_id=f5f0ecaf38e74c96bc439fde75d93d37)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:27:34.247156 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d65fcf9-1cec-42e3-b0a2-3ac8b80f335f admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:27:34.255469 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8247427d-d4b5-48cd-8f2c-dd6777d53536 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:27:34.260335 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658a711e-aba7-41d0-a45d-a9300d551219 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:27:34.264723 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658a711e-aba7-41d0-a45d-a9300d551219 admin admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:27:34.264723 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658a711e-aba7-41d0-a45d-a9300d551219 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:27:34.264723 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658a711e-aba7-41d0-a45d-a9300d551219 admin admin] PATH_INFO: `/v3/projects` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:27:34.264723 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658a711e-aba7-41d0-a45d-a9300d551219 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:27:34.264723 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658a711e-aba7-41d0-a45d-a9300d551219 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:27:34.422975 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1331/2632] 10.222.0.55 () {60 vars in 1153 bytes} [Fri Sep 23 15:27:34 2022] POST /identity/v3/projects => generated 383 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 23 15:27:34.427428 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658a711e-aba7-41d0-a45d-a9300d551219 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:27:34.493643 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658a711e-aba7-41d0-a45d-a9300d551219 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:27:34.498648 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06f3d37-57f5-4053-b8be-812ef9f38b86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:27:34.499260 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06f3d37-57f5-4053-b8be-812ef9f38b86 admin admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:27:34.499616 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06f3d37-57f5-4053-b8be-812ef9f38b86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:27:34.499895 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06f3d37-57f5-4053-b8be-812ef9f38b86 admin admin] PATH_INFO: `/v3/users` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:27:34.500570 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06f3d37-57f5-4053-b8be-812ef9f38b86 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:27:34.502540 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06f3d37-57f5-4053-b8be-812ef9f38b86 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:27:34.527303 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-4d65fcf9-1cec-42e3-b0a2-3ac8b80f335f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f5f0ecaf38e74c96bc439fde75d93d37'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:27:34.540001 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-4d65fcf9-1cec-42e3-b0a2-3ac8b80f335f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f5f0ecaf38e74c96bc439fde75d93d37'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:27:34.546770 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-4d65fcf9-1cec-42e3-b0a2-3ac8b80f335f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f5f0ecaf38e74c96bc439fde75d93d37'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:27:34.662491 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-4d65fcf9-1cec-42e3-b0a2-3ac8b80f335f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f5f0ecaf38e74c96bc439fde75d93d37'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:27:34.663226 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-4d65fcf9-1cec-42e3-b0a2-3ac8b80f335f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f5f0ecaf38e74c96bc439fde75d93d37'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:27:34.670985 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1332/2633] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Sep 23 15:27:34 2022] POST /identity/v3/users => generated 292 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 23 15:27:34.671155 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1302/2634] 10.222.0.55 () {58 vars in 1224 bytes} [Fri Sep 23 15:27:34 2022] DELETE /identity/v3/users/f5f0ecaf38e74c96bc439fde75d93d37 => generated 0 bytes in 510 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:27:34.672726 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06f3d37-57f5-4053-b8be-812ef9f38b86 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:27:34.674400 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-e06f3d37-57f5-4053-b8be-812ef9f38b86 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:27:34.687160 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06f3d37-57f5-4053-b8be-812ef9f38b86 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:27:34.751601 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8481622-c20d-4c2a-a410-1bab4445e87d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:27:34.752265 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8481622-c20d-4c2a-a410-1bab4445e87d admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:27:34.752676 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8481622-c20d-4c2a-a410-1bab4445e87d admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:27:34.752962 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8481622-c20d-4c2a-a410-1bab4445e87d admin admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:27:34.753698 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8481622-c20d-4c2a-a410-1bab4445e87d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:27:34.755454 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8481622-c20d-4c2a-a410-1bab4445e87d admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:27:34.772109 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1333/2635] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Sep 23 15:27:34 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:27:34.775787 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d65fcf9-1cec-42e3-b0a2-3ac8b80f335f admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:27:34.789040 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d65fcf9-1cec-42e3-b0a2-3ac8b80f335f admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:27:34.794062 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e7f1a5-5dbd-4ad3-a901-73d3441c07f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:27:34.794812 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e7f1a5-5dbd-4ad3-a901-73d3441c07f8 admin admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:27:34.795176 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e7f1a5-5dbd-4ad3-a901-73d3441c07f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:27:34.795513 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e7f1a5-5dbd-4ad3-a901-73d3441c07f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:27:34.796260 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e7f1a5-5dbd-4ad3-a901-73d3441c07f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:27:34.798030 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e7f1a5-5dbd-4ad3-a901-73d3441c07f8 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:27:34.811773 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1303/2636] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Sep 23 15:27:34 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 23 15:27:34.813130 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8481622-c20d-4c2a-a410-1bab4445e87d admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:27:34.823604 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8481622-c20d-4c2a-a410-1bab4445e87d admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:27:34.828360 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6774b8a-34ec-4047-a378-c51ae30dc0fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:27:34.829029 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6774b8a-34ec-4047-a378-c51ae30dc0fb admin admin] REQUEST_METHOD: `PUT` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:27:34.829375 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6774b8a-34ec-4047-a378-c51ae30dc0fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:27:34.829580 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6774b8a-34ec-4047-a378-c51ae30dc0fb admin admin] PATH_INFO: `/v3/projects/93df0b4aea37431488a310c6393204de/users/3fd09d24daec43c283eecef93ea5a944/roles/af257be2dee54d6b942f58650196582c` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:27:34.848087 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6774b8a-34ec-4047-a378-c51ae30dc0fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=93df0b4aea37431488a310c6393204de, user_id=3fd09d24daec43c283eecef93ea5a944, role_id=af257be2dee54d6b942f58650196582c)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:27:34.851278 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6774b8a-34ec-4047-a378-c51ae30dc0fb admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:27:34.982202 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1334/2637] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Sep 23 15:27:34 2022] PUT /identity/v3/projects/93df0b4aea37431488a310c6393204de/users/3fd09d24daec43c283eecef93ea5a944/roles/af257be2dee54d6b942f58650196582c => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 15:27:34.993090 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c176a1cb-d586-4909-834f-cee0e0a8b837 None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:27:34.993591 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c176a1cb-d586-4909-834f-cee0e0a8b837 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:27:34.993691 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c176a1cb-d586-4909-834f-cee0e0a8b837 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:27:35.024255 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-c176a1cb-d586-4909-834f-cee0e0a8b837 None None] MFA Rules not processed for user `3fd09d24daec43c283eecef93ea5a944`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:27:35.025510 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-c176a1cb-d586-4909-834f-cee0e0a8b837 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:27:35.053060 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6774b8a-34ec-4047-a378-c51ae30dc0fb admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:27:35.056145 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-d6774b8a-34ec-4047-a378-c51ae30dc0fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:27:35.098332 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1304/2638] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:27:34 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:27:35.102892 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c176a1cb-d586-4909-834f-cee0e0a8b837 None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:27:35.142386 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:27:35.157716 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6774b8a-34ec-4047-a378-c51ae30dc0fb admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:27:35.163098 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb22ef10-f2ba-49e7-8735-25d82e3464bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:27:35.163767 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb22ef10-f2ba-49e7-8735-25d82e3464bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:27:35.164183 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb22ef10-f2ba-49e7-8735-25d82e3464bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:27:35.164530 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb22ef10-f2ba-49e7-8735-25d82e3464bc admin admin] PATH_INFO: `/v3/projects/ab19e9f7cb0e49548a9ab74b9d2b4aec` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:27:35.165633 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb22ef10-f2ba-49e7-8735-25d82e3464bc admin admin] RBAC: Authorizing `identity:delete_project(project_id=ab19e9f7cb0e49548a9ab74b9d2b4aec)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:27:35.168504 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb22ef10-f2ba-49e7-8735-25d82e3464bc admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:27:35.181813 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c176a1cb-d586-4909-834f-cee0e0a8b837 None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:27:35.187764 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2001cc87-71cf-4778-985d-c05ecf00ef32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:27:35.188450 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2001cc87-71cf-4778-985d-c05ecf00ef32 admin admin] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:27:35.188731 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2001cc87-71cf-4778-985d-c05ecf00ef32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:27:35.189004 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2001cc87-71cf-4778-985d-c05ecf00ef32 admin admin] PATH_INFO: `/v3/projects` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:27:35.190312 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2001cc87-71cf-4778-985d-c05ecf00ef32 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:27:35.192625 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2001cc87-71cf-4778-985d-c05ecf00ef32 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:27:35.247705 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:27:35.380746 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1305/2639] 10.222.0.55 () {60 vars in 1153 bytes} [Fri Sep 23 15:27:35 2022] POST /identity/v3/projects => generated 385 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 23 15:27:35.382488 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2001cc87-71cf-4778-985d-c05ecf00ef32 admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:27:35.392739 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-cb22ef10-f2ba-49e7-8735-25d82e3464bc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ab19e9f7cb0e49548a9ab74b9d2b4aec', 'request_id': 'req-cb22ef10-f2ba-49e7-8735-25d82e3464bc'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:27:35.400352 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-cb22ef10-f2ba-49e7-8735-25d82e3464bc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ab19e9f7cb0e49548a9ab74b9d2b4aec', 'request_id': 'req-cb22ef10-f2ba-49e7-8735-25d82e3464bc'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:27:35.406260 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-cb22ef10-f2ba-49e7-8735-25d82e3464bc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ab19e9f7cb0e49548a9ab74b9d2b4aec', 'request_id': 'req-cb22ef10-f2ba-49e7-8735-25d82e3464bc'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:27:35.451150 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2001cc87-71cf-4778-985d-c05ecf00ef32 admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:27:35.455863 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26b0daf-d7b6-4eec-a704-59de3748c79a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:27:35.456507 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26b0daf-d7b6-4eec-a704-59de3748c79a admin admin] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:27:35.456790 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26b0daf-d7b6-4eec-a704-59de3748c79a admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:27:35.457064 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26b0daf-d7b6-4eec-a704-59de3748c79a admin admin] PATH_INFO: `/v3/users` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:27:35.457687 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26b0daf-d7b6-4eec-a704-59de3748c79a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:27:35.459595 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26b0daf-d7b6-4eec-a704-59de3748c79a admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:27:35.621643 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-cb22ef10-f2ba-49e7-8735-25d82e3464bc admin admin] The token cache is being invalidate because project ab19e9f7cb0e49548a9ab74b9d2b4aec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81994) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 15:27:35.621956 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-cb22ef10-f2ba-49e7-8735-25d82e3464bc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:27:35.632045 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-cb22ef10-f2ba-49e7-8735-25d82e3464bc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:27:35.632045 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1335/2640] 10.222.0.55 () {58 vars in 1233 bytes} [Fri Sep 23 15:27:35 2022] DELETE /identity/v3/projects/ab19e9f7cb0e49548a9ab74b9d2b4aec => generated 0 bytes in 572 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 15:27:35.632045 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473dcf45-74ae-46f2-af37-84eb9081f893 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:27:35.632045 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473dcf45-74ae-46f2-af37-84eb9081f893 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:27:35.632045 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473dcf45-74ae-46f2-af37-84eb9081f893 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:27:35.650755 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-473dcf45-74ae-46f2-af37-84eb9081f893 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:27:35.651859 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-473dcf45-74ae-46f2-af37-84eb9081f893 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:27:35.659581 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1336/2641] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:27:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 15:27:35.669484 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce155176-1b56-46f6-be47-b9c7c08c327d None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:27:35.669770 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce155176-1b56-46f6-be47-b9c7c08c327d None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:27:35.670038 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce155176-1b56-46f6-be47-b9c7c08c327d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:27:35.683077 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1306/2642] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Sep 23 15:27:35 2022] POST /identity/v3/users => generated 299 bytes in 300 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:27:35.690555 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-ce155176-1b56-46f6-be47-b9c7c08c327d None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:27:35.691918 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-ce155176-1b56-46f6-be47-b9c7c08c327d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:27:35.692173 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26b0daf-d7b6-4eec-a704-59de3748c79a admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:27:35.696157 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-b26b0daf-d7b6-4eec-a704-59de3748c79a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:27:35.703573 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1337/2643] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:27:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:27:35.705185 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce155176-1b56-46f6-be47-b9c7c08c327d None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:27:35.709436 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26b0daf-d7b6-4eec-a704-59de3748c79a admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:27:35.714337 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84abb321-9415-4d96-a2f5-8e8e25d3bf57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:27:35.715134 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84abb321-9415-4d96-a2f5-8e8e25d3bf57 admin admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:27:35.715449 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce155176-1b56-46f6-be47-b9c7c08c327d None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:27:35.715573 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84abb321-9415-4d96-a2f5-8e8e25d3bf57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:27:35.715864 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84abb321-9415-4d96-a2f5-8e8e25d3bf57 admin admin] PATH_INFO: `/v3/roles` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:27:35.716663 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84abb321-9415-4d96-a2f5-8e8e25d3bf57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:27:35.718308 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84abb321-9415-4d96-a2f5-8e8e25d3bf57 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:27:35.720362 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5a6d17-2f7a-485c-a0a0-9fdbeb5474ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:27:35.721034 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5a6d17-2f7a-485c-a0a0-9fdbeb5474ad admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:27:35.721377 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5a6d17-2f7a-485c-a0a0-9fdbeb5474ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:27:35.721705 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5a6d17-2f7a-485c-a0a0-9fdbeb5474ad admin admin] PATH_INFO: `/v3/domains` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:27:35.722792 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5a6d17-2f7a-485c-a0a0-9fdbeb5474ad admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:27:35.722792 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5a6d17-2f7a-485c-a0a0-9fdbeb5474ad admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:27:35.724773 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5a6d17-2f7a-485c-a0a0-9fdbeb5474ad admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:27:35.735218 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1307/2644] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Sep 23 15:27:35 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:27:35.736630 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84abb321-9415-4d96-a2f5-8e8e25d3bf57 admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:27:35.740335 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1338/2645] 10.222.0.55 () {58 vars in 1153 bytes} [Fri Sep 23 15:27:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 15:27:35.745762 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5a6d17-2f7a-485c-a0a0-9fdbeb5474ad admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:27:35.749441 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84abb321-9415-4d96-a2f5-8e8e25d3bf57 admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:27:35.754471 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30048d1-05e6-4892-9522-58b4af08a556 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:27:35.755151 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30048d1-05e6-4892-9522-58b4af08a556 admin admin] REQUEST_METHOD: `PUT` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:27:35.755435 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30048d1-05e6-4892-9522-58b4af08a556 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:27:35.755827 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30048d1-05e6-4892-9522-58b4af08a556 admin admin] PATH_INFO: `/v3/projects/c080e6c0ff4947cf82d854e457c8e6ad/users/0c7b199fc0ca4698bfa27336a51f00fd/roles/8e2919e79df040e8926f21607818b58c` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:27:35.759090 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5a6d17-2f7a-485c-a0a0-9fdbeb5474ad admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:27:35.780948 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb395b5-9ecc-4d59-9a28-7e386fde7954 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:27:35.780948 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb395b5-9ecc-4d59-9a28-7e386fde7954 admin admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:27:35.780948 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb395b5-9ecc-4d59-9a28-7e386fde7954 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:27:35.780948 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb395b5-9ecc-4d59-9a28-7e386fde7954 admin admin] PATH_INFO: `/v3/projects` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:27:35.781485 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb395b5-9ecc-4d59-9a28-7e386fde7954 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:27:35.781485 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb395b5-9ecc-4d59-9a28-7e386fde7954 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:27:35.781677 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30048d1-05e6-4892-9522-58b4af08a556 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c080e6c0ff4947cf82d854e457c8e6ad, user_id=0c7b199fc0ca4698bfa27336a51f00fd, role_id=8e2919e79df040e8926f21607818b58c)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:27:35.781677 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30048d1-05e6-4892-9522-58b4af08a556 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:27:36.004049 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1308/2646] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Sep 23 15:27:35 2022] PUT /identity/v3/projects/c080e6c0ff4947cf82d854e457c8e6ad/users/0c7b199fc0ca4698bfa27336a51f00fd/roles/8e2919e79df040e8926f21607818b58c => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:27:36.004890 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30048d1-05e6-4892-9522-58b4af08a556 admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:27:36.006418 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-e30048d1-05e6-4892-9522-58b4af08a556 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:27:36.067513 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30048d1-05e6-4892-9522-58b4af08a556 admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:27:36.067513 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c70fd2f-7fbd-4fd7-b919-3a3b3b61d0b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd259178570864ffeb936ee10dfec6b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:27:36.067802 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c70fd2f-7fbd-4fd7-b919-3a3b3b61d0b6 service neutron] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:27:36.067928 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c70fd2f-7fbd-4fd7-b919-3a3b3b61d0b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:27:36.068268 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c70fd2f-7fbd-4fd7-b919-3a3b3b61d0b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:27:36.070403 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-5c70fd2f-7fbd-4fd7-b919-3a3b3b61d0b6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:27:36.080312 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1339/2647] 10.222.0.55 () {60 vars in 1153 bytes} [Fri Sep 23 15:27:35 2022] POST /identity/v3/projects => generated 367 bytes in 335 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 23 15:27:36.081460 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb395b5-9ecc-4d59-9a28-7e386fde7954 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:27:36.144503 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c70fd2f-7fbd-4fd7-b919-3a3b3b61d0b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:27:36.145943 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c70fd2f-7fbd-4fd7-b919-3a3b3b61d0b6 service neutron] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:27:36.148298 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb395b5-9ecc-4d59-9a28-7e386fde7954 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:27:36.153679 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38542066-3396-486d-995f-e51ffdfe13dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:27:36.154288 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38542066-3396-486d-995f-e51ffdfe13dd admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:27:36.154575 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38542066-3396-486d-995f-e51ffdfe13dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:27:36.154859 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38542066-3396-486d-995f-e51ffdfe13dd admin admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:27:36.155502 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38542066-3396-486d-995f-e51ffdfe13dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:27:36.156665 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1309/2648] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:27:36 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:27:36.157480 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38542066-3396-486d-995f-e51ffdfe13dd admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:27:36.157752 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c70fd2f-7fbd-4fd7-b919-3a3b3b61d0b6 service neutron] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:27:36.169822 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1340/2649] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Sep 23 15:27:36 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:27:36.174146 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38542066-3396-486d-995f-e51ffdfe13dd admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:27:36.181025 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:27:36.185510 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38542066-3396-486d-995f-e51ffdfe13dd admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:27:36.190450 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9341da42-b3ec-4ea8-9496-e45250a3f9b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:27:36.191210 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9341da42-b3ec-4ea8-9496-e45250a3f9b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:27:36.191585 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9341da42-b3ec-4ea8-9496-e45250a3f9b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:27:36.191968 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9341da42-b3ec-4ea8-9496-e45250a3f9b2 admin admin] PATH_INFO: `/v3/projects/c080e6c0ff4947cf82d854e457c8e6ad/users/0c7b199fc0ca4698bfa27336a51f00fd/roles/8e2919e79df040e8926f21607818b58c` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:27:36.194279 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9341da42-b3ec-4ea8-9496-e45250a3f9b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c080e6c0ff4947cf82d854e457c8e6ad, user_id=0c7b199fc0ca4698bfa27336a51f00fd, role_id=8e2919e79df040e8926f21607818b58c)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:27:36.197254 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9341da42-b3ec-4ea8-9496-e45250a3f9b2 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:27:36.212719 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1341/2650] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Sep 23 15:27:36 2022] PUT /identity/v3/projects/c080e6c0ff4947cf82d854e457c8e6ad/users/0c7b199fc0ca4698bfa27336a51f00fd/roles/8e2919e79df040e8926f21607818b58c => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:27:36.223281 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a722d7c7-f83e-4fcc-aec6-e6043384c032 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:27:36.223563 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a722d7c7-f83e-4fcc-aec6-e6043384c032 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:27:36.223862 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a722d7c7-f83e-4fcc-aec6-e6043384c032 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:27:36.239959 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:27:36.241934 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c70fd2f-7fbd-4fd7-b919-3a3b3b61d0b6 service neutron] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:27:36.255582 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-a722d7c7-f83e-4fcc-aec6-e6043384c032 None None] MFA Rules not processed for user `0c7b199fc0ca4698bfa27336a51f00fd`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:27:36.257462 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-a722d7c7-f83e-4fcc-aec6-e6043384c032 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:27:36.299665 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cad6672-8884-4475-8899-62e06dfc459c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:27:36.300317 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cad6672-8884-4475-8899-62e06dfc459c admin admin] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:27:36.300600 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cad6672-8884-4475-8899-62e06dfc459c admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:27:36.300868 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cad6672-8884-4475-8899-62e06dfc459c admin admin] PATH_INFO: `/v3/users` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:27:36.301547 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cad6672-8884-4475-8899-62e06dfc459c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:27:36.303503 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cad6672-8884-4475-8899-62e06dfc459c admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:27:36.333195 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1342/2651] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:27:36 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 15:27:36.345556 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a722d7c7-f83e-4fcc-aec6-e6043384c032 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:27:36.362929 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-a722d7c7-f83e-4fcc-aec6-e6043384c032 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:27:36.406974 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a722d7c7-f83e-4fcc-aec6-e6043384c032 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:27:36.411170 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fd4118-50f7-4d7d-bd2c-2cff706c7fc7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feca6278be904ef8b0dc580a67e477f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:27:36.411792 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fd4118-50f7-4d7d-bd2c-2cff706c7fc7 service cinder] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:27:36.412102 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fd4118-50f7-4d7d-bd2c-2cff706c7fc7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:27:36.412381 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fd4118-50f7-4d7d-bd2c-2cff706c7fc7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:27:36.414346 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-e6fd4118-50f7-4d7d-bd2c-2cff706c7fc7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:27:36.475249 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fd4118-50f7-4d7d-bd2c-2cff706c7fc7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:27:36.476760 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fd4118-50f7-4d7d-bd2c-2cff706c7fc7 service cinder] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:27:36.485645 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1343/2652] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:27:36 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 23 15:27:36.513479 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1310/2653] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Sep 23 15:27:36 2022] POST /identity/v3/users => generated 284 bytes in 356 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:27:36.518198 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fd4118-50f7-4d7d-bd2c-2cff706c7fc7 service cinder] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:27:36.529900 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fd4118-50f7-4d7d-bd2c-2cff706c7fc7 service cinder] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:27:36.536707 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8ebd48-46eb-48a9-9129-368f5af66bab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:27:36.537337 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8ebd48-46eb-48a9-9129-368f5af66bab admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:27:36.537651 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8ebd48-46eb-48a9-9129-368f5af66bab admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:27:36.538078 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8ebd48-46eb-48a9-9129-368f5af66bab admin admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:27:36.538858 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8ebd48-46eb-48a9-9129-368f5af66bab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:27:36.540614 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8ebd48-46eb-48a9-9129-368f5af66bab admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:27:36.551749 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1344/2654] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Sep 23 15:27:36 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:27:36.555954 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cad6672-8884-4475-8899-62e06dfc459c admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:27:36.566612 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cad6672-8884-4475-8899-62e06dfc459c admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:27:36.572633 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcbd07c-3e82-4863-a8a8-2f0646d9ef3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:27:36.573337 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcbd07c-3e82-4863-a8a8-2f0646d9ef3d admin admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:27:36.573720 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcbd07c-3e82-4863-a8a8-2f0646d9ef3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:27:36.574068 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcbd07c-3e82-4863-a8a8-2f0646d9ef3d admin admin] PATH_INFO: `/v3/roles` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:27:36.574725 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bcbd07c-3e82-4863-a8a8-2f0646d9ef3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:27:36.576460 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bcbd07c-3e82-4863-a8a8-2f0646d9ef3d admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:27:36.588465 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1311/2655] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Sep 23 15:27:36 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:27:36.593791 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8ebd48-46eb-48a9-9129-368f5af66bab admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:27:36.605548 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8ebd48-46eb-48a9-9129-368f5af66bab admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:27:36.616015 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab3638f-21cb-4006-926e-7a08ca847e6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:27:36.617487 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab3638f-21cb-4006-926e-7a08ca847e6c admin admin] REQUEST_METHOD: `PUT` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:27:36.617786 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab3638f-21cb-4006-926e-7a08ca847e6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:27:36.618175 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab3638f-21cb-4006-926e-7a08ca847e6c admin admin] PATH_INFO: `/v3/projects/da10aefd5449405fa33b562194ac8d64/users/022157e6b332489795a6f3e5e7971329/roles/af257be2dee54d6b942f58650196582c` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:27:36.639606 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab3638f-21cb-4006-926e-7a08ca847e6c admin admin] RBAC: Authorizing `identity:create_grant(project_id=da10aefd5449405fa33b562194ac8d64, user_id=022157e6b332489795a6f3e5e7971329, role_id=af257be2dee54d6b942f58650196582c)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:27:36.642476 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab3638f-21cb-4006-926e-7a08ca847e6c admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:27:36.793290 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1345/2656] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Sep 23 15:27:36 2022] PUT /identity/v3/projects/da10aefd5449405fa33b562194ac8d64/users/022157e6b332489795a6f3e5e7971329/roles/af257be2dee54d6b942f58650196582c => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:27:36.804384 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f166b653-6aea-481a-8118-0aec31cabfd5 None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:27:36.804675 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f166b653-6aea-481a-8118-0aec31cabfd5 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:27:36.804970 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f166b653-6aea-481a-8118-0aec31cabfd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:27:36.834515 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-f166b653-6aea-481a-8118-0aec31cabfd5 None None] MFA Rules not processed for user `022157e6b332489795a6f3e5e7971329`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:27:36.836370 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-f166b653-6aea-481a-8118-0aec31cabfd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:27:36.904570 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:27:36.907285 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:27:36.915799 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1312/2657] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:27:36 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:27:36.926982 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab3638f-21cb-4006-926e-7a08ca847e6c admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:27:36.928698 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-eab3638f-21cb-4006-926e-7a08ca847e6c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:27:36.992292 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab3638f-21cb-4006-926e-7a08ca847e6c admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:27:36.997056 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976fe550-ec7c-41e4-809f-82ac93ed7703 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feca6278be904ef8b0dc580a67e477f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:27:36.997757 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976fe550-ec7c-41e4-809f-82ac93ed7703 service cinder] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:27:36.998201 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976fe550-ec7c-41e4-809f-82ac93ed7703 service cinder] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:27:36.998672 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976fe550-ec7c-41e4-809f-82ac93ed7703 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:27:37.009121 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976fe550-ec7c-41e4-809f-82ac93ed7703 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:27:37.010783 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976fe550-ec7c-41e4-809f-82ac93ed7703 service cinder] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:27:37.020213 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1346/2658] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:27:36 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 23 15:27:38.856832 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f166b653-6aea-481a-8118-0aec31cabfd5 None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:27:38.867318 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f166b653-6aea-481a-8118-0aec31cabfd5 None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:27:38.872222 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff6a0a3-5ae9-4d84-8d23-f8c65e228442 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feca6278be904ef8b0dc580a67e477f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:27:38.872840 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff6a0a3-5ae9-4d84-8d23-f8c65e228442 service cinder] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:27:38.873120 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff6a0a3-5ae9-4d84-8d23-f8c65e228442 service cinder] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:27:38.873411 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff6a0a3-5ae9-4d84-8d23-f8c65e228442 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:27:38.875251 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-bff6a0a3-5ae9-4d84-8d23-f8c65e228442 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:27:38.963920 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff6a0a3-5ae9-4d84-8d23-f8c65e228442 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:27:38.966043 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff6a0a3-5ae9-4d84-8d23-f8c65e228442 service cinder] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:27:38.978142 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1313/2659] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:27:38 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:27:41.131487 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976fe550-ec7c-41e4-809f-82ac93ed7703 service cinder] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:27:41.140472 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976fe550-ec7c-41e4-809f-82ac93ed7703 service cinder] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:27:41.144366 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ea5daf-e87c-48a2-801a-b70293d77102 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feca6278be904ef8b0dc580a67e477f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:27:41.145207 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ea5daf-e87c-48a2-801a-b70293d77102 service cinder] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:27:41.145514 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ea5daf-e87c-48a2-801a-b70293d77102 service cinder] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:27:41.145789 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ea5daf-e87c-48a2-801a-b70293d77102 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:27:41.206922 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ea5daf-e87c-48a2-801a-b70293d77102 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:27:41.208293 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ea5daf-e87c-48a2-801a-b70293d77102 service cinder] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:27:41.218568 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1347/2660] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:27:41 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:27:45.960559 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff6a0a3-5ae9-4d84-8d23-f8c65e228442 service cinder] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:27:46.029819 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff6a0a3-5ae9-4d84-8d23-f8c65e228442 service cinder] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:27:46.034594 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0611bb-d386-429d-9479-0acba6361a74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:27:46.035224 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0611bb-d386-429d-9479-0acba6361a74 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:27:46.035568 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0611bb-d386-429d-9479-0acba6361a74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:27:46.035920 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0611bb-d386-429d-9479-0acba6361a74 admin admin] PATH_INFO: `/v3/users/833c1526634a402b870129ee3f3f008f` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:27:46.038264 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0611bb-d386-429d-9479-0acba6361a74 admin admin] RBAC: Authorizing `identity:delete_user(user_id=833c1526634a402b870129ee3f3f008f)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:27:46.040980 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0611bb-d386-429d-9479-0acba6361a74 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:27:46.317410 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-ae0611bb-d386-429d-9479-0acba6361a74 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '833c1526634a402b870129ee3f3f008f'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:27:46.330201 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-ae0611bb-d386-429d-9479-0acba6361a74 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '833c1526634a402b870129ee3f3f008f'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:27:46.337767 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-ae0611bb-d386-429d-9479-0acba6361a74 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '833c1526634a402b870129ee3f3f008f'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:27:46.437224 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-ae0611bb-d386-429d-9479-0acba6361a74 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '833c1526634a402b870129ee3f3f008f'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:27:46.437851 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-ae0611bb-d386-429d-9479-0acba6361a74 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '833c1526634a402b870129ee3f3f008f'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:27:46.450202 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1314/2661] 10.222.0.55 () {58 vars in 1224 bytes} [Fri Sep 23 15:27:45 2022] DELETE /identity/v3/users/833c1526634a402b870129ee3f3f008f => generated 0 bytes in 486 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:27:46.451369 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4826379-2630-4fb2-ae1f-0d70d2607739 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:27:46.451852 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4826379-2630-4fb2-ae1f-0d70d2607739 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:27:46.451971 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4826379-2630-4fb2-ae1f-0d70d2607739 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:27:46.473036 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-e4826379-2630-4fb2-ae1f-0d70d2607739 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:27:46.474255 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-e4826379-2630-4fb2-ae1f-0d70d2607739 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:27:46.534553 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1348/2662] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:27:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:27:46.544706 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0611bb-d386-429d-9479-0acba6361a74 admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:27:46.546149 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-ae0611bb-d386-429d-9479-0acba6361a74 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:27:46.560558 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:27:46.606282 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0611bb-d386-429d-9479-0acba6361a74 admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:27:46.609709 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25763df6-f506-42f9-9f06-75bb02ed0606 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd259178570864ffeb936ee10dfec6b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:27:46.610308 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25763df6-f506-42f9-9f06-75bb02ed0606 service neutron] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:27:46.610590 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25763df6-f506-42f9-9f06-75bb02ed0606 service neutron] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:27:46.610857 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25763df6-f506-42f9-9f06-75bb02ed0606 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:27:46.620488 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25763df6-f506-42f9-9f06-75bb02ed0606 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:27:46.621907 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25763df6-f506-42f9-9f06-75bb02ed0606 service neutron] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:27:46.631172 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1315/2663] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:27:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:27:47.194138 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4826379-2630-4fb2-ae1f-0d70d2607739 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:27:47.195560 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-e4826379-2630-4fb2-ae1f-0d70d2607739 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:27:47.208253 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4826379-2630-4fb2-ae1f-0d70d2607739 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:27:47.213531 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb41bd83-f173-4da4-a429-82c421a3103b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:27:47.214184 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb41bd83-f173-4da4-a429-82c421a3103b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:27:47.214469 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb41bd83-f173-4da4-a429-82c421a3103b admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:27:47.214744 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb41bd83-f173-4da4-a429-82c421a3103b admin admin] PATH_INFO: `/v3/projects/aaed7382e3b34d449183114c31315ace` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:27:47.215879 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb41bd83-f173-4da4-a429-82c421a3103b admin admin] RBAC: Authorizing `identity:delete_project(project_id=aaed7382e3b34d449183114c31315ace)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:27:47.217812 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb41bd83-f173-4da4-a429-82c421a3103b admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:27:47.383595 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:27:47.398534 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-eb41bd83-f173-4da4-a429-82c421a3103b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aaed7382e3b34d449183114c31315ace', 'request_id': 'req-eb41bd83-f173-4da4-a429-82c421a3103b'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:27:47.405456 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-eb41bd83-f173-4da4-a429-82c421a3103b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aaed7382e3b34d449183114c31315ace', 'request_id': 'req-eb41bd83-f173-4da4-a429-82c421a3103b'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:27:47.410844 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-eb41bd83-f173-4da4-a429-82c421a3103b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aaed7382e3b34d449183114c31315ace', 'request_id': 'req-eb41bd83-f173-4da4-a429-82c421a3103b'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:27:47.529936 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-eb41bd83-f173-4da4-a429-82c421a3103b admin admin] The token cache is being invalidate because project aaed7382e3b34d449183114c31315ace was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81994) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 15:27:47.530613 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-eb41bd83-f173-4da4-a429-82c421a3103b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:27:47.531137 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-eb41bd83-f173-4da4-a429-82c421a3103b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:27:47.532770 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1349/2664] 10.222.0.55 () {58 vars in 1233 bytes} [Fri Sep 23 15:27:47 2022] DELETE /identity/v3/projects/aaed7382e3b34d449183114c31315ace => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:27:47.539961 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25763df6-f506-42f9-9f06-75bb02ed0606 service neutron] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:27:47.547897 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25763df6-f506-42f9-9f06-75bb02ed0606 service neutron] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:27:47.605749 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b52749-feda-467d-b606-2ca7b2909478 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:27:47.606379 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b52749-feda-467d-b606-2ca7b2909478 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:27:47.606656 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b52749-feda-467d-b606-2ca7b2909478 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:27:47.606930 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b52749-feda-467d-b606-2ca7b2909478 admin admin] PATH_INFO: `/v3/users/3fd09d24daec43c283eecef93ea5a944` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:27:47.608100 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b52749-feda-467d-b606-2ca7b2909478 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3fd09d24daec43c283eecef93ea5a944)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:27:47.610010 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b52749-feda-467d-b606-2ca7b2909478 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:27:47.764007 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb41bd83-f173-4da4-a429-82c421a3103b admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:27:47.769069 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-eb41bd83-f173-4da4-a429-82c421a3103b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:27:47.779606 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb41bd83-f173-4da4-a429-82c421a3103b admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:27:47.784320 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7256cac2-ad41-4b6d-ad71-55a7b5f8fa22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:27:47.784951 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7256cac2-ad41-4b6d-ad71-55a7b5f8fa22 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:27:47.785236 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7256cac2-ad41-4b6d-ad71-55a7b5f8fa22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:27:47.785512 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7256cac2-ad41-4b6d-ad71-55a7b5f8fa22 admin admin] PATH_INFO: `/v3/users/022157e6b332489795a6f3e5e7971329` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:27:47.786645 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7256cac2-ad41-4b6d-ad71-55a7b5f8fa22 admin admin] RBAC: Authorizing `identity:delete_user(user_id=022157e6b332489795a6f3e5e7971329)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:27:47.788627 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7256cac2-ad41-4b6d-ad71-55a7b5f8fa22 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:27:47.812774 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-d3b52749-feda-467d-b606-2ca7b2909478 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3fd09d24daec43c283eecef93ea5a944'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:27:47.826328 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-d3b52749-feda-467d-b606-2ca7b2909478 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3fd09d24daec43c283eecef93ea5a944'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:27:47.833460 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-d3b52749-feda-467d-b606-2ca7b2909478 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3fd09d24daec43c283eecef93ea5a944'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:27:48.014191 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-d3b52749-feda-467d-b606-2ca7b2909478 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3fd09d24daec43c283eecef93ea5a944'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:27:48.014760 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-d3b52749-feda-467d-b606-2ca7b2909478 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3fd09d24daec43c283eecef93ea5a944'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:27:48.021283 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1316/2665] 10.222.0.55 () {58 vars in 1224 bytes} [Fri Sep 23 15:27:47 2022] DELETE /identity/v3/users/3fd09d24daec43c283eecef93ea5a944 => generated 0 bytes in 485 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:27:48.127770 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-7256cac2-ad41-4b6d-ad71-55a7b5f8fa22 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '022157e6b332489795a6f3e5e7971329'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:27:48.141770 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-7256cac2-ad41-4b6d-ad71-55a7b5f8fa22 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '022157e6b332489795a6f3e5e7971329'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:27:48.148784 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-7256cac2-ad41-4b6d-ad71-55a7b5f8fa22 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '022157e6b332489795a6f3e5e7971329'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:27:48.289334 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-7256cac2-ad41-4b6d-ad71-55a7b5f8fa22 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '022157e6b332489795a6f3e5e7971329'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:27:48.290962 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-7256cac2-ad41-4b6d-ad71-55a7b5f8fa22 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '022157e6b332489795a6f3e5e7971329'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:27:48.298266 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1350/2666] 10.222.0.55 () {58 vars in 1224 bytes} [Fri Sep 23 15:27:47 2022] DELETE /identity/v3/users/022157e6b332489795a6f3e5e7971329 => generated 0 bytes in 535 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 15:27:48.303643 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186cde9c-b25d-4bf6-86b7-8a7723b75e3c None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:27:48.303643 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186cde9c-b25d-4bf6-86b7-8a7723b75e3c None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:27:48.303643 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186cde9c-b25d-4bf6-86b7-8a7723b75e3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:27:48.324397 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-186cde9c-b25d-4bf6-86b7-8a7723b75e3c None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:27:48.325650 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-186cde9c-b25d-4bf6-86b7-8a7723b75e3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:27:48.360175 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:27:48.393374 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1317/2667] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:27:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:27:48.403330 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7256cac2-ad41-4b6d-ad71-55a7b5f8fa22 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:27:48.404899 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-7256cac2-ad41-4b6d-ad71-55a7b5f8fa22 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:27:48.465903 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7256cac2-ad41-4b6d-ad71-55a7b5f8fa22 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:27:48.469593 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6817cb5-be5d-4a5a-b284-39f32277e22f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd259178570864ffeb936ee10dfec6b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:27:48.470196 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6817cb5-be5d-4a5a-b284-39f32277e22f service neutron] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:27:48.470490 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6817cb5-be5d-4a5a-b284-39f32277e22f service neutron] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:27:48.470757 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6817cb5-be5d-4a5a-b284-39f32277e22f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:27:48.480282 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6817cb5-be5d-4a5a-b284-39f32277e22f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:27:48.481687 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6817cb5-be5d-4a5a-b284-39f32277e22f service neutron] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:27:48.490994 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1351/2668] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:27:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:27:48.519854 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186cde9c-b25d-4bf6-86b7-8a7723b75e3c None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:27:48.521995 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-186cde9c-b25d-4bf6-86b7-8a7723b75e3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:27:48.541537 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186cde9c-b25d-4bf6-86b7-8a7723b75e3c None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:27:48.546722 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb47e9c-2578-4d91-9bba-b1e67be6b87e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:27:48.547672 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb47e9c-2578-4d91-9bba-b1e67be6b87e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:27:48.548194 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb47e9c-2578-4d91-9bba-b1e67be6b87e admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:27:48.548473 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb47e9c-2578-4d91-9bba-b1e67be6b87e admin admin] PATH_INFO: `/v3/projects/93df0b4aea37431488a310c6393204de` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:27:48.549663 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb47e9c-2578-4d91-9bba-b1e67be6b87e admin admin] RBAC: Authorizing `identity:delete_project(project_id=93df0b4aea37431488a310c6393204de)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:27:48.551636 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb47e9c-2578-4d91-9bba-b1e67be6b87e admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:27:48.661943 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-1cb47e9c-2578-4d91-9bba-b1e67be6b87e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '93df0b4aea37431488a310c6393204de', 'request_id': 'req-1cb47e9c-2578-4d91-9bba-b1e67be6b87e'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:27:48.667804 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-1cb47e9c-2578-4d91-9bba-b1e67be6b87e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '93df0b4aea37431488a310c6393204de', 'request_id': 'req-1cb47e9c-2578-4d91-9bba-b1e67be6b87e'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:27:48.740964 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-1cb47e9c-2578-4d91-9bba-b1e67be6b87e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '93df0b4aea37431488a310c6393204de', 'request_id': 'req-1cb47e9c-2578-4d91-9bba-b1e67be6b87e'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:27:48.755690 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-1cb47e9c-2578-4d91-9bba-b1e67be6b87e admin admin] The token cache is being invalidate because project 93df0b4aea37431488a310c6393204de was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81993) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 15:27:48.756057 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-1cb47e9c-2578-4d91-9bba-b1e67be6b87e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:27:48.756711 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-1cb47e9c-2578-4d91-9bba-b1e67be6b87e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:27:48.758236 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1318/2669] 10.222.0.55 () {58 vars in 1233 bytes} [Fri Sep 23 15:27:48 2022] DELETE /identity/v3/projects/93df0b4aea37431488a310c6393204de => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:27:48.762874 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6817cb5-be5d-4a5a-b284-39f32277e22f service neutron] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:27:48.773061 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6817cb5-be5d-4a5a-b284-39f32277e22f service neutron] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:27:48.804706 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:27:48.840231 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8a678f-718c-4840-addf-0ac20bfe5879 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:27:48.840867 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8a678f-718c-4840-addf-0ac20bfe5879 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:27:48.841145 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8a678f-718c-4840-addf-0ac20bfe5879 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:27:48.841424 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8a678f-718c-4840-addf-0ac20bfe5879 admin admin] PATH_INFO: `/v3/users/0c7b199fc0ca4698bfa27336a51f00fd` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:27:48.842585 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8a678f-718c-4840-addf-0ac20bfe5879 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0c7b199fc0ca4698bfa27336a51f00fd)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:27:48.844656 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8a678f-718c-4840-addf-0ac20bfe5879 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:27:48.919704 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb47e9c-2578-4d91-9bba-b1e67be6b87e admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:27:48.921293 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-1cb47e9c-2578-4d91-9bba-b1e67be6b87e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:27:48.935479 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb47e9c-2578-4d91-9bba-b1e67be6b87e admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:27:48.940461 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385679b9-e8fb-45d4-9e8f-3228290635f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:27:48.941094 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385679b9-e8fb-45d4-9e8f-3228290635f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:27:48.941372 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385679b9-e8fb-45d4-9e8f-3228290635f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:27:48.941671 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385679b9-e8fb-45d4-9e8f-3228290635f3 admin admin] PATH_INFO: `/v3/projects/da10aefd5449405fa33b562194ac8d64` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:27:48.942815 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385679b9-e8fb-45d4-9e8f-3228290635f3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=da10aefd5449405fa33b562194ac8d64)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:27:48.944815 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385679b9-e8fb-45d4-9e8f-3228290635f3 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:27:49.081491 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-385679b9-e8fb-45d4-9e8f-3228290635f3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'da10aefd5449405fa33b562194ac8d64', 'request_id': 'req-385679b9-e8fb-45d4-9e8f-3228290635f3'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:27:49.081704 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-df8a678f-718c-4840-addf-0ac20bfe5879 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0c7b199fc0ca4698bfa27336a51f00fd'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:27:49.088236 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-385679b9-e8fb-45d4-9e8f-3228290635f3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'da10aefd5449405fa33b562194ac8d64', 'request_id': 'req-385679b9-e8fb-45d4-9e8f-3228290635f3'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:27:49.095313 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-df8a678f-718c-4840-addf-0ac20bfe5879 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0c7b199fc0ca4698bfa27336a51f00fd'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:27:49.102400 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-df8a678f-718c-4840-addf-0ac20bfe5879 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0c7b199fc0ca4698bfa27336a51f00fd'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:27:49.203407 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-385679b9-e8fb-45d4-9e8f-3228290635f3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'da10aefd5449405fa33b562194ac8d64', 'request_id': 'req-385679b9-e8fb-45d4-9e8f-3228290635f3'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:27:49.216017 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-385679b9-e8fb-45d4-9e8f-3228290635f3 admin admin] The token cache is being invalidate because project da10aefd5449405fa33b562194ac8d64 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81993) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 15:27:49.216316 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-385679b9-e8fb-45d4-9e8f-3228290635f3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:27:49.216973 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-385679b9-e8fb-45d4-9e8f-3228290635f3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:27:49.218490 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1319/2670] 10.222.0.55 () {58 vars in 1233 bytes} [Fri Sep 23 15:27:48 2022] DELETE /identity/v3/projects/da10aefd5449405fa33b562194ac8d64 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:27:49.223293 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b4c573-81db-484a-9640-08123416b038 None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:27:49.223574 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b4c573-81db-484a-9640-08123416b038 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:27:49.223848 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b4c573-81db-484a-9640-08123416b038 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:27:49.245241 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-e2b4c573-81db-484a-9640-08123416b038 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:27:49.246399 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-e2b4c573-81db-484a-9640-08123416b038 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:27:49.280304 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:27:49.307258 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1320/2671] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:27:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 15:27:49.312255 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-df8a678f-718c-4840-addf-0ac20bfe5879 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0c7b199fc0ca4698bfa27336a51f00fd'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:27:49.316687 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-df8a678f-718c-4840-addf-0ac20bfe5879 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0c7b199fc0ca4698bfa27336a51f00fd'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:27:49.316813 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911ebce3-9ee6-4f3e-8ef4-d9f2257a88ed None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:27:49.316813 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911ebce3-9ee6-4f3e-8ef4-d9f2257a88ed None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:27:49.316813 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911ebce3-9ee6-4f3e-8ef4-d9f2257a88ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:27:49.319378 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1352/2672] 10.222.0.55 () {58 vars in 1224 bytes} [Fri Sep 23 15:27:48 2022] DELETE /identity/v3/users/0c7b199fc0ca4698bfa27336a51f00fd => generated 0 bytes in 557 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:27:49.335748 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-911ebce3-9ee6-4f3e-8ef4-d9f2257a88ed None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:27:49.336927 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-911ebce3-9ee6-4f3e-8ef4-d9f2257a88ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:27:49.408284 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1321/2673] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:27:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 15:27:49.409953 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8a678f-718c-4840-addf-0ac20bfe5879 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:27:49.421624 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8a678f-718c-4840-addf-0ac20bfe5879 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:27:49.426307 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69948198-f34b-4be7-a9f2-cb07dd96757d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:27:49.431733 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69948198-f34b-4be7-a9f2-cb07dd96757d admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:27:49.431733 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69948198-f34b-4be7-a9f2-cb07dd96757d admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:27:49.431733 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69948198-f34b-4be7-a9f2-cb07dd96757d admin admin] PATH_INFO: `/v3/domains` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:27:49.431733 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69948198-f34b-4be7-a9f2-cb07dd96757d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:27:49.431733 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69948198-f34b-4be7-a9f2-cb07dd96757d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:27:49.431733 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69948198-f34b-4be7-a9f2-cb07dd96757d admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:27:49.442092 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1353/2674] 10.222.0.55 () {58 vars in 1153 bytes} [Fri Sep 23 15:27:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:27:49.447146 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911ebce3-9ee6-4f3e-8ef4-d9f2257a88ed None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:27:49.456807 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911ebce3-9ee6-4f3e-8ef4-d9f2257a88ed None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:27:49.461611 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2475ce2-f4e4-44bf-aa8b-bdc13a505d08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:27:49.463193 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2475ce2-f4e4-44bf-aa8b-bdc13a505d08 admin admin] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:27:49.466092 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2475ce2-f4e4-44bf-aa8b-bdc13a505d08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:27:49.466092 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2475ce2-f4e4-44bf-aa8b-bdc13a505d08 admin admin] PATH_INFO: `/v3/projects` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:27:49.466092 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2475ce2-f4e4-44bf-aa8b-bdc13a505d08 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:27:49.466546 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2475ce2-f4e4-44bf-aa8b-bdc13a505d08 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:27:49.567153 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1322/2675] 10.222.0.55 () {60 vars in 1153 bytes} [Fri Sep 23 15:27:49 2022] POST /identity/v3/projects => generated 405 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:27:49.573306 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69948198-f34b-4be7-a9f2-cb07dd96757d admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:27:49.584922 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69948198-f34b-4be7-a9f2-cb07dd96757d admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:27:49.646191 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252259ae-b1a1-4359-be9b-716d7a0fa25c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:27:49.646809 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252259ae-b1a1-4359-be9b-716d7a0fa25c admin admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:27:49.647096 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252259ae-b1a1-4359-be9b-716d7a0fa25c admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:27:49.647412 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252259ae-b1a1-4359-be9b-716d7a0fa25c admin admin] PATH_INFO: `/v3/users` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:27:49.648110 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252259ae-b1a1-4359-be9b-716d7a0fa25c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:27:49.650531 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252259ae-b1a1-4359-be9b-716d7a0fa25c admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:27:49.794219 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1354/2676] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Sep 23 15:27:49 2022] POST /identity/v3/users => generated 303 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 23 15:27:49.798889 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2475ce2-f4e4-44bf-aa8b-bdc13a505d08 admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:27:49.806833 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252259ae-b1a1-4359-be9b-716d7a0fa25c admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:27:49.810028 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2475ce2-f4e4-44bf-aa8b-bdc13a505d08 admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:27:49.811660 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-252259ae-b1a1-4359-be9b-716d7a0fa25c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:27:49.815114 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b73ceeb-b7a0-459e-b20f-71ac8708bbe4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:27:49.815731 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b73ceeb-b7a0-459e-b20f-71ac8708bbe4 admin admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:27:49.816035 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b73ceeb-b7a0-459e-b20f-71ac8708bbe4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:27:49.816311 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b73ceeb-b7a0-459e-b20f-71ac8708bbe4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:27:49.817095 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b73ceeb-b7a0-459e-b20f-71ac8708bbe4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:27:49.818690 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b73ceeb-b7a0-459e-b20f-71ac8708bbe4 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:27:49.824531 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252259ae-b1a1-4359-be9b-716d7a0fa25c admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:27:49.829519 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33fb5d5-701a-4d20-b25d-5e4b810bd59c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:27:49.829808 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1323/2677] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Sep 23 15:27:49 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:27:49.833324 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33fb5d5-701a-4d20-b25d-5e4b810bd59c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:27:49.833618 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33fb5d5-701a-4d20-b25d-5e4b810bd59c admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:27:49.838058 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b73ceeb-b7a0-459e-b20f-71ac8708bbe4 admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:27:49.838630 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33fb5d5-701a-4d20-b25d-5e4b810bd59c admin admin] PATH_INFO: `/v3/projects/c080e6c0ff4947cf82d854e457c8e6ad` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:27:49.838630 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33fb5d5-701a-4d20-b25d-5e4b810bd59c admin admin] RBAC: Authorizing `identity:delete_project(project_id=c080e6c0ff4947cf82d854e457c8e6ad)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:27:49.838630 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33fb5d5-701a-4d20-b25d-5e4b810bd59c admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:27:49.844906 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b73ceeb-b7a0-459e-b20f-71ac8708bbe4 admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:27:49.850302 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15dbe5e-aac7-46da-abbe-1339553f97f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:27:49.850900 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15dbe5e-aac7-46da-abbe-1339553f97f8 admin admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:27:49.851165 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15dbe5e-aac7-46da-abbe-1339553f97f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:27:49.851447 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15dbe5e-aac7-46da-abbe-1339553f97f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:27:49.852156 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15dbe5e-aac7-46da-abbe-1339553f97f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:27:49.854265 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15dbe5e-aac7-46da-abbe-1339553f97f8 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:27:49.864268 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:27:49.869308 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1324/2678] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Sep 23 15:27:49 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 23 15:27:49.870194 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15dbe5e-aac7-46da-abbe-1339553f97f8 admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:27:49.879492 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15dbe5e-aac7-46da-abbe-1339553f97f8 admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:27:49.884200 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9896ff-f0a3-45fe-b273-f1642ea54191 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:27:49.885160 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9896ff-f0a3-45fe-b273-f1642ea54191 admin admin] REQUEST_METHOD: `PUT` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:27:49.885487 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9896ff-f0a3-45fe-b273-f1642ea54191 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:27:49.885770 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9896ff-f0a3-45fe-b273-f1642ea54191 admin admin] PATH_INFO: `/v3/projects/f2eb3b48063d4770a05e1d79859230aa/users/6cb3fcf79c4348e197ad2d4111b3ba7d/roles/af257be2dee54d6b942f58650196582c` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:27:49.902300 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9896ff-f0a3-45fe-b273-f1642ea54191 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f2eb3b48063d4770a05e1d79859230aa, user_id=6cb3fcf79c4348e197ad2d4111b3ba7d, role_id=af257be2dee54d6b942f58650196582c)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:27:49.905343 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9896ff-f0a3-45fe-b273-f1642ea54191 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:27:50.031047 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1325/2679] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Sep 23 15:27:49 2022] PUT /identity/v3/projects/f2eb3b48063d4770a05e1d79859230aa/users/6cb3fcf79c4348e197ad2d4111b3ba7d/roles/af257be2dee54d6b942f58650196582c => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 15:27:50.042921 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-f33fb5d5-701a-4d20-b25d-5e4b810bd59c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c080e6c0ff4947cf82d854e457c8e6ad', 'request_id': 'req-f33fb5d5-701a-4d20-b25d-5e4b810bd59c'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:27:50.043908 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66149ce7-2898-47d4-9d1b-99f6c574d812 None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:27:50.044258 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66149ce7-2898-47d4-9d1b-99f6c574d812 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:27:50.044533 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66149ce7-2898-47d4-9d1b-99f6c574d812 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:27:50.051729 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-f33fb5d5-701a-4d20-b25d-5e4b810bd59c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c080e6c0ff4947cf82d854e457c8e6ad', 'request_id': 'req-f33fb5d5-701a-4d20-b25d-5e4b810bd59c'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:27:50.058218 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-f33fb5d5-701a-4d20-b25d-5e4b810bd59c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c080e6c0ff4947cf82d854e457c8e6ad', 'request_id': 'req-f33fb5d5-701a-4d20-b25d-5e4b810bd59c'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:27:50.075824 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-66149ce7-2898-47d4-9d1b-99f6c574d812 None None] MFA Rules not processed for user `6cb3fcf79c4348e197ad2d4111b3ba7d`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:27:50.077251 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-66149ce7-2898-47d4-9d1b-99f6c574d812 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:27:50.144637 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1326/2680] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:27:50 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 15:27:50.149216 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66149ce7-2898-47d4-9d1b-99f6c574d812 None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:27:50.154306 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:27:50.210070 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-f33fb5d5-701a-4d20-b25d-5e4b810bd59c admin admin] The token cache is being invalidate because project c080e6c0ff4947cf82d854e457c8e6ad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81994) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 15:27:50.210512 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-f33fb5d5-701a-4d20-b25d-5e4b810bd59c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:27:50.211167 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-f33fb5d5-701a-4d20-b25d-5e4b810bd59c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:27:50.213613 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1355/2681] 10.222.0.55 () {58 vars in 1233 bytes} [Fri Sep 23 15:27:49 2022] DELETE /identity/v3/projects/c080e6c0ff4947cf82d854e457c8e6ad => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:27:50.214622 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66149ce7-2898-47d4-9d1b-99f6c574d812 None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:27:50.219414 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d83d98-0b44-41bd-8974-af6c339a50bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:27:50.220080 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d83d98-0b44-41bd-8974-af6c339a50bb admin admin] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:27:50.220366 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d83d98-0b44-41bd-8974-af6c339a50bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:27:50.220637 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d83d98-0b44-41bd-8974-af6c339a50bb admin admin] PATH_INFO: `/v3/projects` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:27:50.221284 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d83d98-0b44-41bd-8974-af6c339a50bb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:27:50.223249 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d83d98-0b44-41bd-8974-af6c339a50bb admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:27:50.405310 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1327/2682] 10.222.0.55 () {60 vars in 1153 bytes} [Fri Sep 23 15:27:50 2022] POST /identity/v3/projects => generated 403 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 23 15:27:50.407379 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33fb5d5-701a-4d20-b25d-5e4b810bd59c admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:27:50.409131 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-f33fb5d5-701a-4d20-b25d-5e4b810bd59c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:27:50.422888 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33fb5d5-701a-4d20-b25d-5e4b810bd59c admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:27:50.485704 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a18b98-2ff9-4869-8778-e3e26b59ad63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:27:50.486378 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a18b98-2ff9-4869-8778-e3e26b59ad63 admin admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:27:50.486659 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a18b98-2ff9-4869-8778-e3e26b59ad63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:27:50.486933 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a18b98-2ff9-4869-8778-e3e26b59ad63 admin admin] PATH_INFO: `/v3/users` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:27:50.487702 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a18b98-2ff9-4869-8778-e3e26b59ad63 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:27:50.489628 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a18b98-2ff9-4869-8778-e3e26b59ad63 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:27:50.586173 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1356/2683] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Sep 23 15:27:50 2022] POST /identity/v3/users => generated 308 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 23 15:27:50.591024 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d83d98-0b44-41bd-8974-af6c339a50bb admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:27:50.601624 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d83d98-0b44-41bd-8974-af6c339a50bb admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:27:50.606560 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918df6f5-4842-4be3-9924-f2775d51de97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:27:50.607156 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918df6f5-4842-4be3-9924-f2775d51de97 admin admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:27:50.607436 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918df6f5-4842-4be3-9924-f2775d51de97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:27:50.607791 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918df6f5-4842-4be3-9924-f2775d51de97 admin admin] PATH_INFO: `/v3/roles` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:27:50.608493 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918df6f5-4842-4be3-9924-f2775d51de97 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:27:50.610085 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918df6f5-4842-4be3-9924-f2775d51de97 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:27:50.620455 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1328/2684] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Sep 23 15:27:50 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:27:50.624825 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a18b98-2ff9-4869-8778-e3e26b59ad63 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:27:50.636022 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a18b98-2ff9-4869-8778-e3e26b59ad63 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:27:50.640775 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93913d84-0cb2-40c6-bd31-1af255c5f6a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:27:50.641430 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93913d84-0cb2-40c6-bd31-1af255c5f6a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:27:50.641798 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93913d84-0cb2-40c6-bd31-1af255c5f6a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:27:50.642073 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93913d84-0cb2-40c6-bd31-1af255c5f6a2 admin admin] PATH_INFO: `/v3/projects/4c6b71d136d449c6951565c42a68766d/users/4986a1758d8f45008d9908bff08d1ee9/roles/8e2919e79df040e8926f21607818b58c` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:27:50.662409 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93913d84-0cb2-40c6-bd31-1af255c5f6a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c6b71d136d449c6951565c42a68766d, user_id=4986a1758d8f45008d9908bff08d1ee9, role_id=8e2919e79df040e8926f21607818b58c)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:27:50.662409 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93913d84-0cb2-40c6-bd31-1af255c5f6a2 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:27:50.763387 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1357/2685] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Sep 23 15:27:50 2022] PUT /identity/v3/projects/4c6b71d136d449c6951565c42a68766d/users/4986a1758d8f45008d9908bff08d1ee9/roles/8e2919e79df040e8926f21607818b58c => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:27:50.767538 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918df6f5-4842-4be3-9924-f2775d51de97 admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:27:50.778547 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918df6f5-4842-4be3-9924-f2775d51de97 admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:27:50.838568 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76205555-e777-4579-8751-44c878a39f01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:27:50.839176 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76205555-e777-4579-8751-44c878a39f01 admin admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:27:50.839462 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76205555-e777-4579-8751-44c878a39f01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:27:50.839752 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76205555-e777-4579-8751-44c878a39f01 admin admin] PATH_INFO: `/v3/roles` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:27:50.840591 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76205555-e777-4579-8751-44c878a39f01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:27:50.842253 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76205555-e777-4579-8751-44c878a39f01 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:27:50.851847 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1329/2686] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Sep 23 15:27:50 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:27:50.856980 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93913d84-0cb2-40c6-bd31-1af255c5f6a2 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:27:50.867745 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93913d84-0cb2-40c6-bd31-1af255c5f6a2 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:27:50.873414 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0726e8a0-80a1-4d9b-8ba7-76648b9e268a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:27:50.874071 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0726e8a0-80a1-4d9b-8ba7-76648b9e268a admin admin] REQUEST_METHOD: `PUT` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:27:50.874347 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0726e8a0-80a1-4d9b-8ba7-76648b9e268a admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:27:50.874864 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0726e8a0-80a1-4d9b-8ba7-76648b9e268a admin admin] PATH_INFO: `/v3/projects/4c6b71d136d449c6951565c42a68766d/users/4986a1758d8f45008d9908bff08d1ee9/roles/8e2919e79df040e8926f21607818b58c` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:27:50.876807 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0726e8a0-80a1-4d9b-8ba7-76648b9e268a admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c6b71d136d449c6951565c42a68766d, user_id=4986a1758d8f45008d9908bff08d1ee9, role_id=8e2919e79df040e8926f21607818b58c)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:27:50.879384 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0726e8a0-80a1-4d9b-8ba7-76648b9e268a admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:27:50.890350 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1358/2687] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Sep 23 15:27:50 2022] PUT /identity/v3/projects/4c6b71d136d449c6951565c42a68766d/users/4986a1758d8f45008d9908bff08d1ee9/roles/8e2919e79df040e8926f21607818b58c => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:27:50.902820 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701e02a1-fef8-47e5-b209-4e9a00bf3d13 None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:27:50.903114 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701e02a1-fef8-47e5-b209-4e9a00bf3d13 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:27:50.903395 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701e02a1-fef8-47e5-b209-4e9a00bf3d13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:27:50.932692 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-701e02a1-fef8-47e5-b209-4e9a00bf3d13 None None] MFA Rules not processed for user `4986a1758d8f45008d9908bff08d1ee9`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:27:50.933888 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-701e02a1-fef8-47e5-b209-4e9a00bf3d13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:27:50.999940 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:27:51.005183 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1330/2688] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:27:50 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:27:51.015965 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0726e8a0-80a1-4d9b-8ba7-76648b9e268a admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:27:51.017668 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-0726e8a0-80a1-4d9b-8ba7-76648b9e268a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:27:51.064393 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701e02a1-fef8-47e5-b209-4e9a00bf3d13 None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:27:51.065978 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-701e02a1-fef8-47e5-b209-4e9a00bf3d13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:27:51.076317 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0726e8a0-80a1-4d9b-8ba7-76648b9e268a admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:27:51.086539 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d106b31-7002-4807-b911-260e992fc713 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feca6278be904ef8b0dc580a67e477f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:27:51.087658 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d106b31-7002-4807-b911-260e992fc713 service cinder] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:27:51.087932 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d106b31-7002-4807-b911-260e992fc713 service cinder] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:27:51.088245 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d106b31-7002-4807-b911-260e992fc713 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:27:51.098034 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d106b31-7002-4807-b911-260e992fc713 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:27:51.099438 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d106b31-7002-4807-b911-260e992fc713 service cinder] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:27:51.109619 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1359/2689] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:27:51 2022] GET /identity/v3/auth/tokens => generated 3347 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:27:51.130263 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701e02a1-fef8-47e5-b209-4e9a00bf3d13 None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:27:51.135647 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c19789-c948-4cbb-8c71-32896db99b14 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd259178570864ffeb936ee10dfec6b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:27:51.136407 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c19789-c948-4cbb-8c71-32896db99b14 service neutron] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:27:51.136557 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c19789-c948-4cbb-8c71-32896db99b14 service neutron] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:27:51.136828 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c19789-c948-4cbb-8c71-32896db99b14 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:27:51.138672 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-47c19789-c948-4cbb-8c71-32896db99b14 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:27:51.220590 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c19789-c948-4cbb-8c71-32896db99b14 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:27:51.220590 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c19789-c948-4cbb-8c71-32896db99b14 service neutron] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:27:51.224702 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1331/2690] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:27:51 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:27:51.259814 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d106b31-7002-4807-b911-260e992fc713 service cinder] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:27:51.265614 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:DELETE=7,keystone:SELECT=87,keystone:INSERT=6 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:27:51.330840 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d106b31-7002-4807-b911-260e992fc713 service cinder] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:27:51.335514 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c854e552-1f1a-48d8-9ba0-359d36eb5fee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:27:51.336187 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c854e552-1f1a-48d8-9ba0-359d36eb5fee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:27:51.336470 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c854e552-1f1a-48d8-9ba0-359d36eb5fee admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:27:51.336763 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c854e552-1f1a-48d8-9ba0-359d36eb5fee admin admin] PATH_INFO: `/v3/users/6cb3fcf79c4348e197ad2d4111b3ba7d` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:27:51.338670 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c854e552-1f1a-48d8-9ba0-359d36eb5fee admin admin] RBAC: Authorizing `identity:delete_user(user_id=6cb3fcf79c4348e197ad2d4111b3ba7d)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:27:51.340636 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c854e552-1f1a-48d8-9ba0-359d36eb5fee admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:27:51.640928 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-c854e552-1f1a-48d8-9ba0-359d36eb5fee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6cb3fcf79c4348e197ad2d4111b3ba7d'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:27:51.653260 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-c854e552-1f1a-48d8-9ba0-359d36eb5fee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6cb3fcf79c4348e197ad2d4111b3ba7d'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:27:51.660415 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-c854e552-1f1a-48d8-9ba0-359d36eb5fee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6cb3fcf79c4348e197ad2d4111b3ba7d'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:27:51.786601 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-c854e552-1f1a-48d8-9ba0-359d36eb5fee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6cb3fcf79c4348e197ad2d4111b3ba7d'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:27:51.787225 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-c854e552-1f1a-48d8-9ba0-359d36eb5fee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6cb3fcf79c4348e197ad2d4111b3ba7d'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:27:51.794667 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1360/2691] 10.222.0.55 () {58 vars in 1224 bytes} [Fri Sep 23 15:27:51 2022] DELETE /identity/v3/users/6cb3fcf79c4348e197ad2d4111b3ba7d => generated 0 bytes in 536 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:27:51.799152 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14357e8c-ffe5-49e2-92df-f5360badd25f None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:27:51.799612 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14357e8c-ffe5-49e2-92df-f5360badd25f None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:27:51.799742 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14357e8c-ffe5-49e2-92df-f5360badd25f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:27:51.820153 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-14357e8c-ffe5-49e2-92df-f5360badd25f None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:27:51.820777 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-14357e8c-ffe5-49e2-92df-f5360badd25f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:27:51.885050 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1332/2692] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:27:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:27:51.894459 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c854e552-1f1a-48d8-9ba0-359d36eb5fee admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:27:51.895933 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-c854e552-1f1a-48d8-9ba0-359d36eb5fee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:27:51.964430 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c854e552-1f1a-48d8-9ba0-359d36eb5fee admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:27:51.968465 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fd7aad-d37b-4f23-9b17-800b4fea01c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd259178570864ffeb936ee10dfec6b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:27:51.969162 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fd7aad-d37b-4f23-9b17-800b4fea01c1 service neutron] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:27:51.969388 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fd7aad-d37b-4f23-9b17-800b4fea01c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:27:51.969653 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fd7aad-d37b-4f23-9b17-800b4fea01c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:27:51.980258 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fd7aad-d37b-4f23-9b17-800b4fea01c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:27:51.981768 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fd7aad-d37b-4f23-9b17-800b4fea01c1 service neutron] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:27:51.997359 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1361/2693] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:27:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:27:52.594733 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14357e8c-ffe5-49e2-92df-f5360badd25f None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:27:52.597716 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-14357e8c-ffe5-49e2-92df-f5360badd25f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:27:52.613923 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14357e8c-ffe5-49e2-92df-f5360badd25f None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:27:52.627757 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c33487-dbdb-4d84-a3d0-8655ee280636 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:27:52.628466 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c33487-dbdb-4d84-a3d0-8655ee280636 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:27:52.628744 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c33487-dbdb-4d84-a3d0-8655ee280636 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:27:52.629017 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c33487-dbdb-4d84-a3d0-8655ee280636 admin admin] PATH_INFO: `/v3/projects/f2eb3b48063d4770a05e1d79859230aa` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:27:52.632636 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c33487-dbdb-4d84-a3d0-8655ee280636 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f2eb3b48063d4770a05e1d79859230aa)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:27:52.634587 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c33487-dbdb-4d84-a3d0-8655ee280636 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:27:52.832135 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:27:52.833991 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-81c33487-dbdb-4d84-a3d0-8655ee280636 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f2eb3b48063d4770a05e1d79859230aa', 'request_id': 'req-81c33487-dbdb-4d84-a3d0-8655ee280636'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:27:52.846823 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-81c33487-dbdb-4d84-a3d0-8655ee280636 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f2eb3b48063d4770a05e1d79859230aa', 'request_id': 'req-81c33487-dbdb-4d84-a3d0-8655ee280636'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:27:52.965751 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-81c33487-dbdb-4d84-a3d0-8655ee280636 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f2eb3b48063d4770a05e1d79859230aa', 'request_id': 'req-81c33487-dbdb-4d84-a3d0-8655ee280636'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:27:52.978437 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-81c33487-dbdb-4d84-a3d0-8655ee280636 admin admin] The token cache is being invalidate because project f2eb3b48063d4770a05e1d79859230aa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81993) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 15:27:52.978888 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-81c33487-dbdb-4d84-a3d0-8655ee280636 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:27:52.979595 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-81c33487-dbdb-4d84-a3d0-8655ee280636 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:27:52.981190 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1333/2694] 10.222.0.55 () {58 vars in 1233 bytes} [Fri Sep 23 15:27:52 2022] DELETE /identity/v3/projects/f2eb3b48063d4770a05e1d79859230aa => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:27:52.985333 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fd7aad-d37b-4f23-9b17-800b4fea01c1 service neutron] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:27:52.996371 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fd7aad-d37b-4f23-9b17-800b4fea01c1 service neutron] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:27:53.022060 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:27:53.058385 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2b5e7f-6bdf-4f85-ab92-4d5637eb05d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:27:53.059131 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2b5e7f-6bdf-4f85-ab92-4d5637eb05d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:27:53.066541 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2b5e7f-6bdf-4f85-ab92-4d5637eb05d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:27:53.066541 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2b5e7f-6bdf-4f85-ab92-4d5637eb05d2 admin admin] PATH_INFO: `/v3/users/4986a1758d8f45008d9908bff08d1ee9` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:27:53.066541 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2b5e7f-6bdf-4f85-ab92-4d5637eb05d2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4986a1758d8f45008d9908bff08d1ee9)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:27:53.066541 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2b5e7f-6bdf-4f85-ab92-4d5637eb05d2 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:27:53.360602 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-6d2b5e7f-6bdf-4f85-ab92-4d5637eb05d2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4986a1758d8f45008d9908bff08d1ee9'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:27:53.373683 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-6d2b5e7f-6bdf-4f85-ab92-4d5637eb05d2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4986a1758d8f45008d9908bff08d1ee9'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:27:53.380781 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-6d2b5e7f-6bdf-4f85-ab92-4d5637eb05d2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4986a1758d8f45008d9908bff08d1ee9'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:27:53.517428 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-6d2b5e7f-6bdf-4f85-ab92-4d5637eb05d2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4986a1758d8f45008d9908bff08d1ee9'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:27:53.518051 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-6d2b5e7f-6bdf-4f85-ab92-4d5637eb05d2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4986a1758d8f45008d9908bff08d1ee9'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:27:53.524303 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1362/2695] 10.222.0.55 () {58 vars in 1224 bytes} [Fri Sep 23 15:27:52 2022] DELETE /identity/v3/users/4986a1758d8f45008d9908bff08d1ee9 => generated 0 bytes in 540 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:27:54.006403 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c33487-dbdb-4d84-a3d0-8655ee280636 admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:27:54.014094 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-81c33487-dbdb-4d84-a3d0-8655ee280636 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:27:54.029262 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c33487-dbdb-4d84-a3d0-8655ee280636 admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:27:54.253874 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695340ce-9d74-42eb-a99b-3911008482ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:27:54.254520 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695340ce-9d74-42eb-a99b-3911008482ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:27:54.254798 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695340ce-9d74-42eb-a99b-3911008482ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:27:54.255075 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695340ce-9d74-42eb-a99b-3911008482ba admin admin] PATH_INFO: `/v3/projects/4c6b71d136d449c6951565c42a68766d` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:27:54.256250 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695340ce-9d74-42eb-a99b-3911008482ba admin admin] RBAC: Authorizing `identity:delete_project(project_id=4c6b71d136d449c6951565c42a68766d)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:27:54.258166 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695340ce-9d74-42eb-a99b-3911008482ba admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:27:54.394089 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-695340ce-9d74-42eb-a99b-3911008482ba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4c6b71d136d449c6951565c42a68766d', 'request_id': 'req-695340ce-9d74-42eb-a99b-3911008482ba'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:27:54.400213 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-695340ce-9d74-42eb-a99b-3911008482ba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4c6b71d136d449c6951565c42a68766d', 'request_id': 'req-695340ce-9d74-42eb-a99b-3911008482ba'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:27:54.488907 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-695340ce-9d74-42eb-a99b-3911008482ba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4c6b71d136d449c6951565c42a68766d', 'request_id': 'req-695340ce-9d74-42eb-a99b-3911008482ba'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:27:54.502026 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-695340ce-9d74-42eb-a99b-3911008482ba admin admin] The token cache is being invalidate because project 4c6b71d136d449c6951565c42a68766d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81993) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 15:27:54.502318 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-695340ce-9d74-42eb-a99b-3911008482ba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:27:54.507281 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-695340ce-9d74-42eb-a99b-3911008482ba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:27:54.507281 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1334/2696] 10.222.0.55 () {58 vars in 1233 bytes} [Fri Sep 23 15:27:54 2022] DELETE /identity/v3/projects/4c6b71d136d449c6951565c42a68766d => generated 0 bytes in 499 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:27:54.509872 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9643ad-866f-4dae-b58f-e31e17ac2eb4 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:27:54.510413 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9643ad-866f-4dae-b58f-e31e17ac2eb4 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:27:54.510413 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9643ad-866f-4dae-b58f-e31e17ac2eb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:27:54.530487 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-9b9643ad-866f-4dae-b58f-e31e17ac2eb4 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:27:54.531711 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-9b9643ad-866f-4dae-b58f-e31e17ac2eb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:27:54.591397 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1363/2697] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:27:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:27:54.599427 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30130a1e-3325-4612-98e4-d1d8bb59df99 None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:27:54.599707 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30130a1e-3325-4612-98e4-d1d8bb59df99 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:27:54.600057 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30130a1e-3325-4612-98e4-d1d8bb59df99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:27:54.621432 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-30130a1e-3325-4612-98e4-d1d8bb59df99 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:27:54.622707 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-30130a1e-3325-4612-98e4-d1d8bb59df99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:27:54.631419 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1335/2698] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:27:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:27:54.637733 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9643ad-866f-4dae-b58f-e31e17ac2eb4 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:27:54.647365 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9643ad-866f-4dae-b58f-e31e17ac2eb4 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:27:54.653668 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354211ee-9364-4313-9d3e-863a9e2e05cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:27:54.653978 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354211ee-9364-4313-9d3e-863a9e2e05cd admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:27:54.654264 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354211ee-9364-4313-9d3e-863a9e2e05cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:27:54.654581 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354211ee-9364-4313-9d3e-863a9e2e05cd admin admin] PATH_INFO: `/v3/domains` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:27:54.655187 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354211ee-9364-4313-9d3e-863a9e2e05cd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:27:54.655491 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354211ee-9364-4313-9d3e-863a9e2e05cd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:27:54.657122 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354211ee-9364-4313-9d3e-863a9e2e05cd admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:27:54.669951 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1364/2699] 10.222.0.55 () {58 vars in 1153 bytes} [Fri Sep 23 15:27:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:27:54.674508 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30130a1e-3325-4612-98e4-d1d8bb59df99 None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:27:54.684106 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30130a1e-3325-4612-98e4-d1d8bb59df99 None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:27:54.688921 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cce2382-48fc-4907-b4ff-4ea2780efd20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:27:54.689514 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cce2382-48fc-4907-b4ff-4ea2780efd20 admin admin] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:27:54.689793 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cce2382-48fc-4907-b4ff-4ea2780efd20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:27:54.690065 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cce2382-48fc-4907-b4ff-4ea2780efd20 admin admin] PATH_INFO: `/v3/projects` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:27:54.690716 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cce2382-48fc-4907-b4ff-4ea2780efd20 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:27:54.692684 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cce2382-48fc-4907-b4ff-4ea2780efd20 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:27:54.805093 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1336/2700] 10.222.0.55 () {60 vars in 1153 bytes} [Fri Sep 23 15:27:54 2022] POST /identity/v3/projects => generated 377 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:27:54.809418 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354211ee-9364-4313-9d3e-863a9e2e05cd admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:27:54.819723 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354211ee-9364-4313-9d3e-863a9e2e05cd admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:27:54.851707 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:27:54.878721 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8368f6f3-0d7f-450e-9c22-184960a30e65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:27:54.879355 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8368f6f3-0d7f-450e-9c22-184960a30e65 admin admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:27:54.879642 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8368f6f3-0d7f-450e-9c22-184960a30e65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:27:54.879918 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8368f6f3-0d7f-450e-9c22-184960a30e65 admin admin] PATH_INFO: `/v3/users` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:27:54.880575 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8368f6f3-0d7f-450e-9c22-184960a30e65 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:27:54.882552 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8368f6f3-0d7f-450e-9c22-184960a30e65 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:27:55.028034 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1365/2701] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Sep 23 15:27:54 2022] POST /identity/v3/users => generated 289 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:27:55.032577 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cce2382-48fc-4907-b4ff-4ea2780efd20 admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:27:55.042975 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cce2382-48fc-4907-b4ff-4ea2780efd20 admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:27:55.047562 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7661f3-5b7f-4224-9e6a-1afb56c1a23e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:27:55.048268 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7661f3-5b7f-4224-9e6a-1afb56c1a23e admin admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:27:55.048549 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7661f3-5b7f-4224-9e6a-1afb56c1a23e admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:27:55.048821 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7661f3-5b7f-4224-9e6a-1afb56c1a23e admin admin] PATH_INFO: `/v3/roles` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:27:55.049458 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7661f3-5b7f-4224-9e6a-1afb56c1a23e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:27:55.051041 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7661f3-5b7f-4224-9e6a-1afb56c1a23e admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:27:55.067263 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1337/2702] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Sep 23 15:27:55 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:27:55.067446 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8368f6f3-0d7f-450e-9c22-184960a30e65 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:27:55.077945 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8368f6f3-0d7f-450e-9c22-184960a30e65 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:27:55.082814 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d13bca-c68d-429d-9259-20108f1e0701 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:27:55.083427 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d13bca-c68d-429d-9259-20108f1e0701 admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:27:55.083714 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d13bca-c68d-429d-9259-20108f1e0701 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:27:55.084033 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d13bca-c68d-429d-9259-20108f1e0701 admin admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:27:55.084678 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d13bca-c68d-429d-9259-20108f1e0701 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:27:55.086304 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d13bca-c68d-429d-9259-20108f1e0701 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:27:55.099022 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1366/2703] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Sep 23 15:27:55 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:27:55.104687 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7661f3-5b7f-4224-9e6a-1afb56c1a23e admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:27:55.114638 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7661f3-5b7f-4224-9e6a-1afb56c1a23e admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:27:55.119759 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d050bf86-cc35-4c9e-9de5-291358224d8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:27:55.120448 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d050bf86-cc35-4c9e-9de5-291358224d8a admin admin] REQUEST_METHOD: `PUT` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:27:55.120726 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d050bf86-cc35-4c9e-9de5-291358224d8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:27:55.120999 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d050bf86-cc35-4c9e-9de5-291358224d8a admin admin] PATH_INFO: `/v3/projects/49071443fa3a45aea240e3d43c631c19/users/cb70d8daf7f64d39acd63f6ca03ef69d/roles/af257be2dee54d6b942f58650196582c` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:27:55.139181 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d050bf86-cc35-4c9e-9de5-291358224d8a admin admin] RBAC: Authorizing `identity:create_grant(project_id=49071443fa3a45aea240e3d43c631c19, user_id=cb70d8daf7f64d39acd63f6ca03ef69d, role_id=af257be2dee54d6b942f58650196582c)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:27:55.139630 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d050bf86-cc35-4c9e-9de5-291358224d8a admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:27:55.238272 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1338/2704] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Sep 23 15:27:55 2022] PUT /identity/v3/projects/49071443fa3a45aea240e3d43c631c19/users/cb70d8daf7f64d39acd63f6ca03ef69d/roles/af257be2dee54d6b942f58650196582c => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:27:55.250071 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74851d06-6a2b-44d7-b5eb-39f6c623ee4d None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:27:55.250542 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74851d06-6a2b-44d7-b5eb-39f6c623ee4d None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:27:55.250879 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74851d06-6a2b-44d7-b5eb-39f6c623ee4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:27:55.283275 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-74851d06-6a2b-44d7-b5eb-39f6c623ee4d None None] MFA Rules not processed for user `cb70d8daf7f64d39acd63f6ca03ef69d`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:27:55.284749 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-74851d06-6a2b-44d7-b5eb-39f6c623ee4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:27:55.375076 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1367/2705] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:27:55 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:27:55.379278 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d050bf86-cc35-4c9e-9de5-291358224d8a admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:27:55.386742 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:27:55.452478 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d050bf86-cc35-4c9e-9de5-291358224d8a admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:27:55.457168 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226b42ff-b755-4292-bfe6-e8baaf098499 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:27:55.457747 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226b42ff-b755-4292-bfe6-e8baaf098499 admin admin] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:27:55.458022 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226b42ff-b755-4292-bfe6-e8baaf098499 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:27:55.458350 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226b42ff-b755-4292-bfe6-e8baaf098499 admin admin] PATH_INFO: `/v3/projects` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:27:55.458997 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226b42ff-b755-4292-bfe6-e8baaf098499 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:27:55.460994 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226b42ff-b755-4292-bfe6-e8baaf098499 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:27:55.581535 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1339/2706] 10.222.0.55 () {60 vars in 1153 bytes} [Fri Sep 23 15:27:55 2022] POST /identity/v3/projects => generated 381 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 23 15:27:55.581734 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74851d06-6a2b-44d7-b5eb-39f6c623ee4d None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:27:55.593712 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74851d06-6a2b-44d7-b5eb-39f6c623ee4d None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:27:55.656732 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcfbe8e-9524-4e08-8044-44486e420185 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:27:55.657397 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcfbe8e-9524-4e08-8044-44486e420185 admin admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:27:55.657683 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcfbe8e-9524-4e08-8044-44486e420185 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:27:55.657965 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcfbe8e-9524-4e08-8044-44486e420185 admin admin] PATH_INFO: `/v3/users` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:27:55.658629 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bcfbe8e-9524-4e08-8044-44486e420185 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:27:55.660627 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bcfbe8e-9524-4e08-8044-44486e420185 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:27:55.822046 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1368/2707] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Sep 23 15:27:55 2022] POST /identity/v3/users => generated 297 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:27:55.826389 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226b42ff-b755-4292-bfe6-e8baaf098499 admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:27:55.837952 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226b42ff-b755-4292-bfe6-e8baaf098499 admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:27:55.842430 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195ed4e8-fe45-4d1e-9c45-d2381cf52322 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:27:55.843032 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195ed4e8-fe45-4d1e-9c45-d2381cf52322 admin admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:27:55.843311 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195ed4e8-fe45-4d1e-9c45-d2381cf52322 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:27:55.843586 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195ed4e8-fe45-4d1e-9c45-d2381cf52322 admin admin] PATH_INFO: `/v3/roles` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:27:55.844267 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195ed4e8-fe45-4d1e-9c45-d2381cf52322 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:27:55.845866 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195ed4e8-fe45-4d1e-9c45-d2381cf52322 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:27:55.856643 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1340/2708] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Sep 23 15:27:55 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:27:55.861368 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcfbe8e-9524-4e08-8044-44486e420185 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:27:55.872445 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcfbe8e-9524-4e08-8044-44486e420185 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:27:55.877164 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d884191c-7ddd-4305-9376-d3bb74c4db8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:27:55.877830 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d884191c-7ddd-4305-9376-d3bb74c4db8a admin admin] REQUEST_METHOD: `PUT` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:27:55.878115 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d884191c-7ddd-4305-9376-d3bb74c4db8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:27:55.878491 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d884191c-7ddd-4305-9376-d3bb74c4db8a admin admin] PATH_INFO: `/v3/projects/e160ca98f8f44fcf8637ad1cdcfc2f21/users/22b1619e848244ebb6de03a24382cae9/roles/8e2919e79df040e8926f21607818b58c` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:27:55.895081 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d884191c-7ddd-4305-9376-d3bb74c4db8a admin admin] RBAC: Authorizing `identity:create_grant(project_id=e160ca98f8f44fcf8637ad1cdcfc2f21, user_id=22b1619e848244ebb6de03a24382cae9, role_id=8e2919e79df040e8926f21607818b58c)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:27:55.897773 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d884191c-7ddd-4305-9376-d3bb74c4db8a admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:27:55.970624 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1369/2709] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Sep 23 15:27:55 2022] PUT /identity/v3/projects/e160ca98f8f44fcf8637ad1cdcfc2f21/users/22b1619e848244ebb6de03a24382cae9/roles/8e2919e79df040e8926f21607818b58c => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:27:55.974661 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195ed4e8-fe45-4d1e-9c45-d2381cf52322 admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:27:55.986403 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195ed4e8-fe45-4d1e-9c45-d2381cf52322 admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:27:56.047125 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe187b23-29d1-4f27-a4ab-6c14dec8d432 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:27:56.047831 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe187b23-29d1-4f27-a4ab-6c14dec8d432 admin admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:27:56.048030 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe187b23-29d1-4f27-a4ab-6c14dec8d432 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:27:56.048308 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe187b23-29d1-4f27-a4ab-6c14dec8d432 admin admin] PATH_INFO: `/v3/roles` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:27:56.048944 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe187b23-29d1-4f27-a4ab-6c14dec8d432 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:27:56.050522 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe187b23-29d1-4f27-a4ab-6c14dec8d432 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:27:56.066800 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1341/2710] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Sep 23 15:27:55 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 23 15:27:56.071537 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d884191c-7ddd-4305-9376-d3bb74c4db8a admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:27:56.085052 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d884191c-7ddd-4305-9376-d3bb74c4db8a admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:27:56.090067 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ecfd2e-7dd3-415a-ae04-4e08f19011c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:27:56.090780 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ecfd2e-7dd3-415a-ae04-4e08f19011c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:27:56.091059 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ecfd2e-7dd3-415a-ae04-4e08f19011c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:27:56.091332 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ecfd2e-7dd3-415a-ae04-4e08f19011c0 admin admin] PATH_INFO: `/v3/projects/e160ca98f8f44fcf8637ad1cdcfc2f21/users/22b1619e848244ebb6de03a24382cae9/roles/8e2919e79df040e8926f21607818b58c` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:27:56.093284 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ecfd2e-7dd3-415a-ae04-4e08f19011c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e160ca98f8f44fcf8637ad1cdcfc2f21, user_id=22b1619e848244ebb6de03a24382cae9, role_id=8e2919e79df040e8926f21607818b58c)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:27:56.095999 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ecfd2e-7dd3-415a-ae04-4e08f19011c0 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:27:56.098438 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:27:56.107849 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1370/2711] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Sep 23 15:27:56 2022] PUT /identity/v3/projects/e160ca98f8f44fcf8637ad1cdcfc2f21/users/22b1619e848244ebb6de03a24382cae9/roles/8e2919e79df040e8926f21607818b58c => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:27:56.128950 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebc0093-55d7-4ee5-990c-193a1830f2b4 None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:27:56.128950 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebc0093-55d7-4ee5-990c-193a1830f2b4 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:27:56.128950 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebc0093-55d7-4ee5-990c-193a1830f2b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:27:56.159748 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-9ebc0093-55d7-4ee5-990c-193a1830f2b4 None None] MFA Rules not processed for user `22b1619e848244ebb6de03a24382cae9`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:27:56.161175 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-9ebc0093-55d7-4ee5-990c-193a1830f2b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:27:56.232757 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:27:56.238264 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1342/2712] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:27:56 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:27:56.250814 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ecfd2e-7dd3-415a-ae04-4e08f19011c0 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:27:56.252730 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-45ecfd2e-7dd3-415a-ae04-4e08f19011c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:27:56.319997 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ecfd2e-7dd3-415a-ae04-4e08f19011c0 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:27:56.324988 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bb736d-b9ea-422c-ad02-d84783faa52f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feca6278be904ef8b0dc580a67e477f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:27:56.325613 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bb736d-b9ea-422c-ad02-d84783faa52f service cinder] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:27:56.325898 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bb736d-b9ea-422c-ad02-d84783faa52f service cinder] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:27:56.326172 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bb736d-b9ea-422c-ad02-d84783faa52f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:27:56.336575 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bb736d-b9ea-422c-ad02-d84783faa52f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:27:56.338179 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bb736d-b9ea-422c-ad02-d84783faa52f service cinder] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:27:56.349051 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1371/2713] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:27:56 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:27:56.583942 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebc0093-55d7-4ee5-990c-193a1830f2b4 None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:27:56.596397 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebc0093-55d7-4ee5-990c-193a1830f2b4 None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:27:56.600492 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c22a1c9-cdfd-4778-9003-9f1c3ce02281 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feca6278be904ef8b0dc580a67e477f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:27:56.601050 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c22a1c9-cdfd-4778-9003-9f1c3ce02281 service cinder] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:27:56.601330 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c22a1c9-cdfd-4778-9003-9f1c3ce02281 service cinder] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:27:56.601603 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c22a1c9-cdfd-4778-9003-9f1c3ce02281 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:27:56.671675 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c22a1c9-cdfd-4778-9003-9f1c3ce02281 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:27:56.673196 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c22a1c9-cdfd-4778-9003-9f1c3ce02281 service cinder] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:27:56.683428 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1343/2714] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:27:56 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:28:00.464184 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bb736d-b9ea-422c-ad02-d84783faa52f service cinder] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:28:00.466369 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-78bb736d-b9ea-422c-ad02-d84783faa52f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:28:00.539252 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bb736d-b9ea-422c-ad02-d84783faa52f service cinder] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:28:00.543197 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8323e3-b1ed-49f5-a8b3-f31c475e45d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd259178570864ffeb936ee10dfec6b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:28:00.543876 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8323e3-b1ed-49f5-a8b3-f31c475e45d3 service neutron] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:28:00.544195 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8323e3-b1ed-49f5-a8b3-f31c475e45d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:28:00.544460 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8323e3-b1ed-49f5-a8b3-f31c475e45d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:28:00.546191 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-ec8323e3-b1ed-49f5-a8b3-f31c475e45d3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:28:00.610607 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8323e3-b1ed-49f5-a8b3-f31c475e45d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:28:00.612690 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8323e3-b1ed-49f5-a8b3-f31c475e45d3 service neutron] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:28:00.623484 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1372/2715] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:28:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:28:05.197413 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c22a1c9-cdfd-4778-9003-9f1c3ce02281 service cinder] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:28:05.197413 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-2c22a1c9-cdfd-4778-9003-9f1c3ce02281 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:28:05.225022 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c22a1c9-cdfd-4778-9003-9f1c3ce02281 service cinder] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:28:05.232174 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85746c6-43a4-4909-a8a8-3a19a89a1d23 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd259178570864ffeb936ee10dfec6b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:28:05.232380 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85746c6-43a4-4909-a8a8-3a19a89a1d23 service neutron] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:28:05.232651 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85746c6-43a4-4909-a8a8-3a19a89a1d23 service neutron] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:28:05.232925 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85746c6-43a4-4909-a8a8-3a19a89a1d23 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:28:05.234721 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-b85746c6-43a4-4909-a8a8-3a19a89a1d23 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:28:05.258526 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85746c6-43a4-4909-a8a8-3a19a89a1d23 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:28:05.261317 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85746c6-43a4-4909-a8a8-3a19a89a1d23 service neutron] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:28:05.292594 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1344/2716] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:28:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:28:07.727172 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8323e3-b1ed-49f5-a8b3-f31c475e45d3 service neutron] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:28:07.729887 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-ec8323e3-b1ed-49f5-a8b3-f31c475e45d3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:28:07.831047 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:28:07.896658 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8323e3-b1ed-49f5-a8b3-f31c475e45d3 service neutron] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:28:07.902606 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c56aee7-9130-4c02-be8b-f4628a58990e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09f1a55d180642c781a6e0b017d9f0aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:28:07.903382 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c56aee7-9130-4c02-be8b-f4628a58990e service placement] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:28:07.903698 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c56aee7-9130-4c02-be8b-f4628a58990e service placement] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:28:07.903999 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c56aee7-9130-4c02-be8b-f4628a58990e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:28:07.905944 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-4c56aee7-9130-4c02-be8b-f4628a58990e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:28:07.919702 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c56aee7-9130-4c02-be8b-f4628a58990e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:28:07.921375 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c56aee7-9130-4c02-be8b-f4628a58990e service placement] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:28:07.933405 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1373/2717] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:28:07 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:28:09.465992 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85746c6-43a4-4909-a8a8-3a19a89a1d23 service neutron] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:28:09.476761 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85746c6-43a4-4909-a8a8-3a19a89a1d23 service neutron] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:28:09.482178 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68c772d-8eac-46f3-86bd-04a01ee37159 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:28:09.482872 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68c772d-8eac-46f3-86bd-04a01ee37159 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:28:09.483342 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68c772d-8eac-46f3-86bd-04a01ee37159 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:28:09.483800 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68c772d-8eac-46f3-86bd-04a01ee37159 admin admin] PATH_INFO: `/v3/users/cb70d8daf7f64d39acd63f6ca03ef69d` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:28:09.485009 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68c772d-8eac-46f3-86bd-04a01ee37159 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cb70d8daf7f64d39acd63f6ca03ef69d)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:28:09.487061 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68c772d-8eac-46f3-86bd-04a01ee37159 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:28:09.964262 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c56aee7-9130-4c02-be8b-f4628a58990e service placement] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:28:09.965848 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-4c56aee7-9130-4c02-be8b-f4628a58990e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:28:09.981150 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c56aee7-9130-4c02-be8b-f4628a58990e service placement] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:28:09.986621 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989358e4-5a66-44f9-8232-3771fc1dddf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:28:09.987590 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989358e4-5a66-44f9-8232-3771fc1dddf1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:28:09.987590 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989358e4-5a66-44f9-8232-3771fc1dddf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:28:09.987804 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989358e4-5a66-44f9-8232-3771fc1dddf1 admin admin] PATH_INFO: `/v3/users/4bdaecd5dd5d461383878c4b96ef98be` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:28:09.989188 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989358e4-5a66-44f9-8232-3771fc1dddf1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4bdaecd5dd5d461383878c4b96ef98be)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:28:09.992314 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989358e4-5a66-44f9-8232-3771fc1dddf1 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:28:10.041847 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-f68c772d-8eac-46f3-86bd-04a01ee37159 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cb70d8daf7f64d39acd63f6ca03ef69d'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:28:10.072011 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-f68c772d-8eac-46f3-86bd-04a01ee37159 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cb70d8daf7f64d39acd63f6ca03ef69d'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:28:10.106100 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-f68c772d-8eac-46f3-86bd-04a01ee37159 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cb70d8daf7f64d39acd63f6ca03ef69d'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:28:10.360259 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-f68c772d-8eac-46f3-86bd-04a01ee37159 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cb70d8daf7f64d39acd63f6ca03ef69d'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:28:10.360259 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-f68c772d-8eac-46f3-86bd-04a01ee37159 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cb70d8daf7f64d39acd63f6ca03ef69d'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:28:10.365219 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1345/2718] 10.222.0.55 () {58 vars in 1224 bytes} [Fri Sep 23 15:28:09 2022] DELETE /identity/v3/users/cb70d8daf7f64d39acd63f6ca03ef69d => generated 0 bytes in 900 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:28:10.371702 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c463d1-3698-4325-8b28-da0674d42d33 None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:28:10.371998 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c463d1-3698-4325-8b28-da0674d42d33 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:28:10.372287 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c463d1-3698-4325-8b28-da0674d42d33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:28:10.375780 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-989358e4-5a66-44f9-8232-3771fc1dddf1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4bdaecd5dd5d461383878c4b96ef98be'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:28:10.393216 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-989358e4-5a66-44f9-8232-3771fc1dddf1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4bdaecd5dd5d461383878c4b96ef98be'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:28:10.398445 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-a7c463d1-3698-4325-8b28-da0674d42d33 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:28:10.398445 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-a7c463d1-3698-4325-8b28-da0674d42d33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:28:10.404640 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-989358e4-5a66-44f9-8232-3771fc1dddf1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4bdaecd5dd5d461383878c4b96ef98be'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:28:10.452660 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:28:10.466408 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1346/2719] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:28:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:28:10.480016 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c463d1-3698-4325-8b28-da0674d42d33 None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:28:10.481832 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-a7c463d1-3698-4325-8b28-da0674d42d33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:28:10.499944 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-989358e4-5a66-44f9-8232-3771fc1dddf1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4bdaecd5dd5d461383878c4b96ef98be'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:28:10.505050 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-989358e4-5a66-44f9-8232-3771fc1dddf1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4bdaecd5dd5d461383878c4b96ef98be'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:28:10.511230 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1374/2720] 10.222.0.55 () {58 vars in 1224 bytes} [Fri Sep 23 15:28:09 2022] DELETE /identity/v3/users/4bdaecd5dd5d461383878c4b96ef98be => generated 0 bytes in 544 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 15:28:10.581810 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c463d1-3698-4325-8b28-da0674d42d33 None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:28:10.640875 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2c888a-afe8-4147-932a-f3b3aa7723ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd259178570864ffeb936ee10dfec6b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:28:10.641636 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2c888a-afe8-4147-932a-f3b3aa7723ca service neutron] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:28:10.641985 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2c888a-afe8-4147-932a-f3b3aa7723ca service neutron] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:28:10.642385 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2c888a-afe8-4147-932a-f3b3aa7723ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:28:10.651173 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-be2c888a-afe8-4147-932a-f3b3aa7723ca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:28:10.726291 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2c888a-afe8-4147-932a-f3b3aa7723ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:28:10.727943 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2c888a-afe8-4147-932a-f3b3aa7723ca service neutron] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:28:10.740478 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1347/2721] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:28:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 261 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:28:10.845039 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989358e4-5a66-44f9-8232-3771fc1dddf1 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:28:10.846264 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-989358e4-5a66-44f9-8232-3771fc1dddf1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:28:10.846621 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2c888a-afe8-4147-932a-f3b3aa7723ca service neutron] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:28:10.849317 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-be2c888a-afe8-4147-932a-f3b3aa7723ca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:28:10.862606 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2c888a-afe8-4147-932a-f3b3aa7723ca service neutron] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:28:10.863116 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989358e4-5a66-44f9-8232-3771fc1dddf1 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:28:10.867825 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecd6bc9-68d8-48ed-a461-9de55ad77cb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:28:10.870822 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecd6bc9-68d8-48ed-a461-9de55ad77cb8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:28:10.870822 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecd6bc9-68d8-48ed-a461-9de55ad77cb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:28:10.870822 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecd6bc9-68d8-48ed-a461-9de55ad77cb8 admin admin] PATH_INFO: `/v3/projects/7c95b871ff2b4ed89195a9c0fa94e8b4` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:28:10.871248 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b57a02-60ce-4a1e-af92-c568f2ab053e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:28:10.871357 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ecd6bc9-68d8-48ed-a461-9de55ad77cb8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7c95b871ff2b4ed89195a9c0fa94e8b4)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:28:10.871862 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b57a02-60ce-4a1e-af92-c568f2ab053e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:28:10.872167 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b57a02-60ce-4a1e-af92-c568f2ab053e admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:28:10.872658 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ecd6bc9-68d8-48ed-a461-9de55ad77cb8 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:28:10.872835 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b57a02-60ce-4a1e-af92-c568f2ab053e admin admin] PATH_INFO: `/v3/users/42dc378333ae4bbdbd42ce041ba15d3f` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:28:10.874012 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b57a02-60ce-4a1e-af92-c568f2ab053e admin admin] RBAC: Authorizing `identity:delete_user(user_id=42dc378333ae4bbdbd42ce041ba15d3f)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:28:10.875995 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b57a02-60ce-4a1e-af92-c568f2ab053e admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:28:11.171821 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-6ecd6bc9-68d8-48ed-a461-9de55ad77cb8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7c95b871ff2b4ed89195a9c0fa94e8b4', 'request_id': 'req-6ecd6bc9-68d8-48ed-a461-9de55ad77cb8'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:28:11.177928 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-6ecd6bc9-68d8-48ed-a461-9de55ad77cb8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7c95b871ff2b4ed89195a9c0fa94e8b4', 'request_id': 'req-6ecd6bc9-68d8-48ed-a461-9de55ad77cb8'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:28:11.184079 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-74b57a02-60ce-4a1e-af92-c568f2ab053e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '42dc378333ae4bbdbd42ce041ba15d3f'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:28:11.198279 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-74b57a02-60ce-4a1e-af92-c568f2ab053e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '42dc378333ae4bbdbd42ce041ba15d3f'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:28:11.218816 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-74b57a02-60ce-4a1e-af92-c568f2ab053e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '42dc378333ae4bbdbd42ce041ba15d3f'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:28:11.306172 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-6ecd6bc9-68d8-48ed-a461-9de55ad77cb8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7c95b871ff2b4ed89195a9c0fa94e8b4', 'request_id': 'req-6ecd6bc9-68d8-48ed-a461-9de55ad77cb8'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:28:11.318337 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-6ecd6bc9-68d8-48ed-a461-9de55ad77cb8 admin admin] The token cache is being invalidate because project 7c95b871ff2b4ed89195a9c0fa94e8b4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81993) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 15:28:11.321983 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-6ecd6bc9-68d8-48ed-a461-9de55ad77cb8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:28:11.321983 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-6ecd6bc9-68d8-48ed-a461-9de55ad77cb8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:28:11.321983 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1348/2722] 10.222.0.55 () {58 vars in 1233 bytes} [Fri Sep 23 15:28:10 2022] DELETE /identity/v3/projects/7c95b871ff2b4ed89195a9c0fa94e8b4 => generated 0 bytes in 476 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:28:11.322600 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecd6bc9-68d8-48ed-a461-9de55ad77cb8 admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:28:11.323957 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-6ecd6bc9-68d8-48ed-a461-9de55ad77cb8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:28:11.389598 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecd6bc9-68d8-48ed-a461-9de55ad77cb8 admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:28:11.394598 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05821a9b-a948-4309-9cb3-ff9c82de68d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:28:11.395170 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05821a9b-a948-4309-9cb3-ff9c82de68d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:28:11.395451 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05821a9b-a948-4309-9cb3-ff9c82de68d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:28:11.395724 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05821a9b-a948-4309-9cb3-ff9c82de68d7 admin admin] PATH_INFO: `/v3/projects/49071443fa3a45aea240e3d43c631c19` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:28:11.396880 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05821a9b-a948-4309-9cb3-ff9c82de68d7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=49071443fa3a45aea240e3d43c631c19)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:28:11.398826 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05821a9b-a948-4309-9cb3-ff9c82de68d7 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:28:11.402887 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-74b57a02-60ce-4a1e-af92-c568f2ab053e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '42dc378333ae4bbdbd42ce041ba15d3f'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:28:11.403509 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-74b57a02-60ce-4a1e-af92-c568f2ab053e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '42dc378333ae4bbdbd42ce041ba15d3f'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:28:11.405982 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:28:11.411008 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1375/2723] 10.222.0.55 () {58 vars in 1224 bytes} [Fri Sep 23 15:28:10 2022] DELETE /identity/v3/users/42dc378333ae4bbdbd42ce041ba15d3f => generated 0 bytes in 567 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:28:11.411894 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b57a02-60ce-4a1e-af92-c568f2ab053e admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:28:11.413579 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-74b57a02-60ce-4a1e-af92-c568f2ab053e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:28:11.483898 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b57a02-60ce-4a1e-af92-c568f2ab053e admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:28:11.488725 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146841a0-c54e-447b-99e8-62156b55e90d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:28:11.489362 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146841a0-c54e-447b-99e8-62156b55e90d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:28:11.489633 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146841a0-c54e-447b-99e8-62156b55e90d admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:28:11.489903 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146841a0-c54e-447b-99e8-62156b55e90d admin admin] PATH_INFO: `/v3/users/3a349a48d97d4d0283d630c55b36bd18` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:28:11.491637 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146841a0-c54e-447b-99e8-62156b55e90d admin admin] RBAC: Authorizing `identity:delete_user(user_id=3a349a48d97d4d0283d630c55b36bd18)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:28:11.495078 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146841a0-c54e-447b-99e8-62156b55e90d admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:28:11.593094 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-05821a9b-a948-4309-9cb3-ff9c82de68d7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '49071443fa3a45aea240e3d43c631c19', 'request_id': 'req-05821a9b-a948-4309-9cb3-ff9c82de68d7'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:28:11.599087 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-05821a9b-a948-4309-9cb3-ff9c82de68d7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '49071443fa3a45aea240e3d43c631c19', 'request_id': 'req-05821a9b-a948-4309-9cb3-ff9c82de68d7'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:28:11.757457 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-05821a9b-a948-4309-9cb3-ff9c82de68d7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '49071443fa3a45aea240e3d43c631c19', 'request_id': 'req-05821a9b-a948-4309-9cb3-ff9c82de68d7'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:28:11.759758 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:28:11.770777 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-05821a9b-a948-4309-9cb3-ff9c82de68d7 admin admin] The token cache is being invalidate because project 49071443fa3a45aea240e3d43c631c19 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81993) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 15:28:11.771067 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-05821a9b-a948-4309-9cb3-ff9c82de68d7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:28:11.771724 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-05821a9b-a948-4309-9cb3-ff9c82de68d7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:28:11.773673 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1349/2724] 10.222.0.55 () {58 vars in 1233 bytes} [Fri Sep 23 15:28:11 2022] DELETE /identity/v3/projects/49071443fa3a45aea240e3d43c631c19 => generated 0 bytes in 452 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:28:11.774591 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05821a9b-a948-4309-9cb3-ff9c82de68d7 admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:28:11.775884 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-05821a9b-a948-4309-9cb3-ff9c82de68d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:28:11.800286 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-146841a0-c54e-447b-99e8-62156b55e90d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3a349a48d97d4d0283d630c55b36bd18'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:28:11.814945 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-146841a0-c54e-447b-99e8-62156b55e90d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3a349a48d97d4d0283d630c55b36bd18'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:28:11.822169 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-146841a0-c54e-447b-99e8-62156b55e90d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3a349a48d97d4d0283d630c55b36bd18'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:28:11.842638 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05821a9b-a948-4309-9cb3-ff9c82de68d7 admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:28:11.847531 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a16c798-c07b-4020-80c7-b97f7c0141ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:28:11.848420 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a16c798-c07b-4020-80c7-b97f7c0141ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:28:11.848704 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a16c798-c07b-4020-80c7-b97f7c0141ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:28:11.848986 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a16c798-c07b-4020-80c7-b97f7c0141ce admin admin] PATH_INFO: `/v3/projects/1679242155f94d48871525425f74d1c3` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:28:11.850271 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a16c798-c07b-4020-80c7-b97f7c0141ce admin admin] RBAC: Authorizing `identity:delete_project(project_id=1679242155f94d48871525425f74d1c3)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:28:11.852237 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a16c798-c07b-4020-80c7-b97f7c0141ce admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:28:11.894251 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-146841a0-c54e-447b-99e8-62156b55e90d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3a349a48d97d4d0283d630c55b36bd18'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:28:11.900168 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-146841a0-c54e-447b-99e8-62156b55e90d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3a349a48d97d4d0283d630c55b36bd18'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:28:11.902037 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1376/2725] 10.222.0.55 () {58 vars in 1224 bytes} [Fri Sep 23 15:28:11 2022] DELETE /identity/v3/users/3a349a48d97d4d0283d630c55b36bd18 => generated 0 bytes in 490 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:28:11.905157 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146841a0-c54e-447b-99e8-62156b55e90d admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:28:11.906283 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-146841a0-c54e-447b-99e8-62156b55e90d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:28:11.967823 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-2a16c798-c07b-4020-80c7-b97f7c0141ce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1679242155f94d48871525425f74d1c3', 'request_id': 'req-2a16c798-c07b-4020-80c7-b97f7c0141ce'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:28:11.974323 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-2a16c798-c07b-4020-80c7-b97f7c0141ce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1679242155f94d48871525425f74d1c3', 'request_id': 'req-2a16c798-c07b-4020-80c7-b97f7c0141ce'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:28:11.978625 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146841a0-c54e-447b-99e8-62156b55e90d admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:28:12.036610 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6322deb-b905-4e92-9133-45951a660bcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:28:12.037268 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6322deb-b905-4e92-9133-45951a660bcb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:28:12.037557 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6322deb-b905-4e92-9133-45951a660bcb admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:28:12.037879 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6322deb-b905-4e92-9133-45951a660bcb admin admin] PATH_INFO: `/v3/users/22b1619e848244ebb6de03a24382cae9` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:28:12.039079 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6322deb-b905-4e92-9133-45951a660bcb admin admin] RBAC: Authorizing `identity:delete_user(user_id=22b1619e848244ebb6de03a24382cae9)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:28:12.041178 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6322deb-b905-4e92-9133-45951a660bcb admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:28:12.047743 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-2a16c798-c07b-4020-80c7-b97f7c0141ce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1679242155f94d48871525425f74d1c3', 'request_id': 'req-2a16c798-c07b-4020-80c7-b97f7c0141ce'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:28:12.060765 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-2a16c798-c07b-4020-80c7-b97f7c0141ce admin admin] The token cache is being invalidate because project 1679242155f94d48871525425f74d1c3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81993) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 15:28:12.061088 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-2a16c798-c07b-4020-80c7-b97f7c0141ce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:28:12.061748 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-2a16c798-c07b-4020-80c7-b97f7c0141ce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:28:12.063307 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1350/2726] 10.222.0.55 () {58 vars in 1233 bytes} [Fri Sep 23 15:28:11 2022] DELETE /identity/v3/projects/1679242155f94d48871525425f74d1c3 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:28:12.068388 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de0138c-d497-4081-bf0d-ba636ab8bcf6 None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:28:12.068763 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de0138c-d497-4081-bf0d-ba636ab8bcf6 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:28:12.069103 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de0138c-d497-4081-bf0d-ba636ab8bcf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:28:12.093762 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-0de0138c-d497-4081-bf0d-ba636ab8bcf6 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:28:12.094951 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-0de0138c-d497-4081-bf0d-ba636ab8bcf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:28:12.102817 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1351/2727] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:28:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:28:12.111039 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4f8b4c-588f-4a87-a9d5-a48d85f6680c None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:28:12.111320 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4f8b4c-588f-4a87-a9d5-a48d85f6680c None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:28:12.111594 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4f8b4c-588f-4a87-a9d5-a48d85f6680c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:28:12.132848 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-5c4f8b4c-588f-4a87-a9d5-a48d85f6680c None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:28:12.147105 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-5c4f8b4c-588f-4a87-a9d5-a48d85f6680c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:28:12.147105 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1352/2728] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:28:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 15:28:12.150086 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4f8b4c-588f-4a87-a9d5-a48d85f6680c None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:28:12.159193 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4f8b4c-588f-4a87-a9d5-a48d85f6680c None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:28:12.164028 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee921a9b-7a2d-4338-95d3-9803e7b9d8ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:28:12.164625 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee921a9b-7a2d-4338-95d3-9803e7b9d8ed admin admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:28:12.164908 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee921a9b-7a2d-4338-95d3-9803e7b9d8ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:28:12.165721 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee921a9b-7a2d-4338-95d3-9803e7b9d8ed admin admin] PATH_INFO: `/v3/domains` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:28:12.166815 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee921a9b-7a2d-4338-95d3-9803e7b9d8ed admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:28:12.167206 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee921a9b-7a2d-4338-95d3-9803e7b9d8ed admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:28:12.170010 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee921a9b-7a2d-4338-95d3-9803e7b9d8ed admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:28:12.186215 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1353/2729] 10.222.0.55 () {58 vars in 1153 bytes} [Fri Sep 23 15:28:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 15:28:12.186215 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee921a9b-7a2d-4338-95d3-9803e7b9d8ed admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:28:12.186215 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-ee921a9b-7a2d-4338-95d3-9803e7b9d8ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:28:12.198915 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee921a9b-7a2d-4338-95d3-9803e7b9d8ed admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:28:12.204520 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab656538-bfe3-464d-bc12-f6c0d7e38692 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:28:12.204520 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab656538-bfe3-464d-bc12-f6c0d7e38692 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:28:12.205348 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab656538-bfe3-464d-bc12-f6c0d7e38692 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:28:12.205348 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab656538-bfe3-464d-bc12-f6c0d7e38692 admin admin] PATH_INFO: `/v3/projects/ce652035804f47dc803c040c20704414` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:28:12.206328 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab656538-bfe3-464d-bc12-f6c0d7e38692 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ce652035804f47dc803c040c20704414)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:28:12.208203 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab656538-bfe3-464d-bc12-f6c0d7e38692 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:28:12.247031 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-d6322deb-b905-4e92-9133-45951a660bcb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '22b1619e848244ebb6de03a24382cae9'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:28:12.260938 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-d6322deb-b905-4e92-9133-45951a660bcb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '22b1619e848244ebb6de03a24382cae9'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:28:12.268335 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-d6322deb-b905-4e92-9133-45951a660bcb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '22b1619e848244ebb6de03a24382cae9'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:28:12.328259 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-ab656538-bfe3-464d-bc12-f6c0d7e38692 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ce652035804f47dc803c040c20704414', 'request_id': 'req-ab656538-bfe3-464d-bc12-f6c0d7e38692'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:28:12.333897 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-ab656538-bfe3-464d-bc12-f6c0d7e38692 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ce652035804f47dc803c040c20704414', 'request_id': 'req-ab656538-bfe3-464d-bc12-f6c0d7e38692'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:28:12.334251 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-d6322deb-b905-4e92-9133-45951a660bcb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '22b1619e848244ebb6de03a24382cae9'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:28:12.334936 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-d6322deb-b905-4e92-9133-45951a660bcb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '22b1619e848244ebb6de03a24382cae9'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:28:12.342554 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1377/2730] 10.222.0.55 () {58 vars in 1224 bytes} [Fri Sep 23 15:28:11 2022] DELETE /identity/v3/users/22b1619e848244ebb6de03a24382cae9 => generated 0 bytes in 440 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:28:12.343344 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6322deb-b905-4e92-9133-45951a660bcb admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:28:12.344724 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-d6322deb-b905-4e92-9133-45951a660bcb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:28:12.409101 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6322deb-b905-4e92-9133-45951a660bcb admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:28:12.413795 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63070e51-c213-4eef-9467-c51ec67c3d3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:28:12.415519 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63070e51-c213-4eef-9467-c51ec67c3d3f admin admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:28:12.415907 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63070e51-c213-4eef-9467-c51ec67c3d3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:28:12.416206 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63070e51-c213-4eef-9467-c51ec67c3d3f admin admin] PATH_INFO: `/v3/projects` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:28:12.416832 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63070e51-c213-4eef-9467-c51ec67c3d3f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:28:12.418806 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63070e51-c213-4eef-9467-c51ec67c3d3f admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:28:12.459920 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-ab656538-bfe3-464d-bc12-f6c0d7e38692 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ce652035804f47dc803c040c20704414', 'request_id': 'req-ab656538-bfe3-464d-bc12-f6c0d7e38692'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:28:12.472344 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-ab656538-bfe3-464d-bc12-f6c0d7e38692 admin admin] The token cache is being invalidate because project ce652035804f47dc803c040c20704414 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81993) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 15:28:12.472640 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-ab656538-bfe3-464d-bc12-f6c0d7e38692 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:28:12.473300 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-ab656538-bfe3-464d-bc12-f6c0d7e38692 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:28:12.474770 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1354/2731] 10.222.0.55 () {58 vars in 1233 bytes} [Fri Sep 23 15:28:12 2022] DELETE /identity/v3/projects/ce652035804f47dc803c040c20704414 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:28:12.488742 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:28:12.571032 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1378/2732] 10.222.0.55 () {60 vars in 1153 bytes} [Fri Sep 23 15:28:12 2022] POST /identity/v3/projects => generated 385 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:28:12.573101 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab656538-bfe3-464d-bc12-f6c0d7e38692 admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:28:12.574405 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:28:12.574722 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-ab656538-bfe3-464d-bc12-f6c0d7e38692 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:28:12.686037 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab656538-bfe3-464d-bc12-f6c0d7e38692 admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:28:12.758552 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089d47e6-7152-4055-b3e8-e7dabee7ffb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:28:12.759882 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089d47e6-7152-4055-b3e8-e7dabee7ffb2 admin admin] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:28:12.760229 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089d47e6-7152-4055-b3e8-e7dabee7ffb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:28:12.760514 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089d47e6-7152-4055-b3e8-e7dabee7ffb2 admin admin] PATH_INFO: `/v3/users` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:28:12.761198 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089d47e6-7152-4055-b3e8-e7dabee7ffb2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:28:12.763159 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089d47e6-7152-4055-b3e8-e7dabee7ffb2 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:28:12.861680 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63070e51-c213-4eef-9467-c51ec67c3d3f admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:28:12.863607 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-63070e51-c213-4eef-9467-c51ec67c3d3f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:28:12.879634 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63070e51-c213-4eef-9467-c51ec67c3d3f admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:28:12.885094 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1355/2733] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Sep 23 15:28:12 2022] POST /identity/v3/users => generated 293 bytes in 313 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:28:12.885540 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b54479b-4b89-49d1-a5cf-26e3dec8b130 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:28:12.886188 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b54479b-4b89-49d1-a5cf-26e3dec8b130 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:28:12.886473 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b54479b-4b89-49d1-a5cf-26e3dec8b130 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:28:12.886829 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b54479b-4b89-49d1-a5cf-26e3dec8b130 admin admin] PATH_INFO: `/v3/projects/e160ca98f8f44fcf8637ad1cdcfc2f21` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:28:12.888994 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b54479b-4b89-49d1-a5cf-26e3dec8b130 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e160ca98f8f44fcf8637ad1cdcfc2f21)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:28:12.889122 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089d47e6-7152-4055-b3e8-e7dabee7ffb2 admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:28:12.891294 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b54479b-4b89-49d1-a5cf-26e3dec8b130 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:28:12.899535 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089d47e6-7152-4055-b3e8-e7dabee7ffb2 admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:28:12.904411 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fbc373-080d-46dd-8be4-d37ec5f77527 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:28:12.905023 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fbc373-080d-46dd-8be4-d37ec5f77527 admin admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:28:12.906956 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fbc373-080d-46dd-8be4-d37ec5f77527 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:28:12.906956 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fbc373-080d-46dd-8be4-d37ec5f77527 admin admin] PATH_INFO: `/v3/roles` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:28:12.906956 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fbc373-080d-46dd-8be4-d37ec5f77527 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:28:12.907868 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fbc373-080d-46dd-8be4-d37ec5f77527 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:28:12.919275 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1356/2734] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Sep 23 15:28:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 23 15:28:12.923308 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fbc373-080d-46dd-8be4-d37ec5f77527 admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:28:12.933936 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fbc373-080d-46dd-8be4-d37ec5f77527 admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:28:12.938694 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e1e68e-84c7-4b1a-be33-36f76dc33b67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:28:12.939292 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e1e68e-84c7-4b1a-be33-36f76dc33b67 admin admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:28:12.939572 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e1e68e-84c7-4b1a-be33-36f76dc33b67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:28:12.939845 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e1e68e-84c7-4b1a-be33-36f76dc33b67 admin admin] PATH_INFO: `/v3/roles` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:28:12.940507 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e1e68e-84c7-4b1a-be33-36f76dc33b67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:28:12.942081 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e1e68e-84c7-4b1a-be33-36f76dc33b67 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:28:12.952897 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1357/2735] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Sep 23 15:28:12 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 23 15:28:12.957112 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e1e68e-84c7-4b1a-be33-36f76dc33b67 admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:28:12.967873 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e1e68e-84c7-4b1a-be33-36f76dc33b67 admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:28:12.972511 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce97440-26a2-4fc2-a1c5-7b69023d6f7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:28:12.973569 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce97440-26a2-4fc2-a1c5-7b69023d6f7b admin admin] REQUEST_METHOD: `PUT` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:28:12.978169 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce97440-26a2-4fc2-a1c5-7b69023d6f7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:28:12.978169 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce97440-26a2-4fc2-a1c5-7b69023d6f7b admin admin] PATH_INFO: `/v3/projects/695b6a2a390b481bbca2fd367273c268/users/97e06913feae4c23a9abbcd508f818d3/roles/af257be2dee54d6b942f58650196582c` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:28:12.983481 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-5b54479b-4b89-49d1-a5cf-26e3dec8b130 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e160ca98f8f44fcf8637ad1cdcfc2f21', 'request_id': 'req-5b54479b-4b89-49d1-a5cf-26e3dec8b130'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:28:12.990187 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce97440-26a2-4fc2-a1c5-7b69023d6f7b admin admin] RBAC: Authorizing `identity:create_grant(project_id=695b6a2a390b481bbca2fd367273c268, user_id=97e06913feae4c23a9abbcd508f818d3, role_id=af257be2dee54d6b942f58650196582c)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:28:12.990672 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-5b54479b-4b89-49d1-a5cf-26e3dec8b130 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e160ca98f8f44fcf8637ad1cdcfc2f21', 'request_id': 'req-5b54479b-4b89-49d1-a5cf-26e3dec8b130'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:28:12.992874 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce97440-26a2-4fc2-a1c5-7b69023d6f7b admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:28:12.997930 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-5b54479b-4b89-49d1-a5cf-26e3dec8b130 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e160ca98f8f44fcf8637ad1cdcfc2f21', 'request_id': 'req-5b54479b-4b89-49d1-a5cf-26e3dec8b130'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:28:13.071141 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1358/2736] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Sep 23 15:28:12 2022] PUT /identity/v3/projects/695b6a2a390b481bbca2fd367273c268/users/97e06913feae4c23a9abbcd508f818d3/roles/af257be2dee54d6b942f58650196582c => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 15:28:13.076257 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd5d073-6c02-4ef7-9483-d5ee848d9593 None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:28:13.076549 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd5d073-6c02-4ef7-9483-d5ee848d9593 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:28:13.076819 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd5d073-6c02-4ef7-9483-d5ee848d9593 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:28:13.095708 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-2fd5d073-6c02-4ef7-9483-d5ee848d9593 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:28:13.096940 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-2fd5d073-6c02-4ef7-9483-d5ee848d9593 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:28:13.116656 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-5b54479b-4b89-49d1-a5cf-26e3dec8b130 admin admin] The token cache is being invalidate because project e160ca98f8f44fcf8637ad1cdcfc2f21 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81994) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 15:28:13.120102 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-5b54479b-4b89-49d1-a5cf-26e3dec8b130 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:28:13.120102 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-5b54479b-4b89-49d1-a5cf-26e3dec8b130 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:28:13.120733 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1379/2737] 10.222.0.55 () {58 vars in 1233 bytes} [Fri Sep 23 15:28:12 2022] DELETE /identity/v3/projects/e160ca98f8f44fcf8637ad1cdcfc2f21 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 15:28:13.125690 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1fe00b-8b69-4782-9449-5ae71666a539 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:28:13.125970 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1fe00b-8b69-4782-9449-5ae71666a539 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:28:13.126240 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1fe00b-8b69-4782-9449-5ae71666a539 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:28:13.146227 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-4b1fe00b-8b69-4782-9449-5ae71666a539 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:28:13.147437 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-4b1fe00b-8b69-4782-9449-5ae71666a539 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:28:13.159596 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1359/2738] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:28:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 15:28:13.170133 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd5d073-6c02-4ef7-9483-d5ee848d9593 None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:28:13.171778 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-2fd5d073-6c02-4ef7-9483-d5ee848d9593 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:28:13.202130 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:28:13.217811 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1380/2739] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:28:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 15:28:13.227406 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3649af2-a0ec-44e9-894b-9bc85614df85 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:28:13.230592 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3649af2-a0ec-44e9-894b-9bc85614df85 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:28:13.230859 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3649af2-a0ec-44e9-894b-9bc85614df85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:28:13.235513 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:28:13.250191 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd5d073-6c02-4ef7-9483-d5ee848d9593 None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:28:13.254654 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49307894-0b91-4dd5-8267-f9e25299c240 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd259178570864ffeb936ee10dfec6b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:28:13.255258 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49307894-0b91-4dd5-8267-f9e25299c240 service neutron] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:28:13.255535 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49307894-0b91-4dd5-8267-f9e25299c240 service neutron] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:28:13.255800 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49307894-0b91-4dd5-8267-f9e25299c240 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:28:13.256049 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-a3649af2-a0ec-44e9-894b-9bc85614df85 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:28:13.257379 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-a3649af2-a0ec-44e9-894b-9bc85614df85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:28:13.266933 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1381/2740] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:28:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:28:13.268088 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49307894-0b91-4dd5-8267-f9e25299c240 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:28:13.269722 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49307894-0b91-4dd5-8267-f9e25299c240 service neutron] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:28:13.271386 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3649af2-a0ec-44e9-894b-9bc85614df85 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:28:13.281610 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3649af2-a0ec-44e9-894b-9bc85614df85 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:28:13.283137 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1360/2741] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:28:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 23 15:28:13.287099 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91d97d4-7bb2-41be-95b0-d60eb73e58be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:28:13.287886 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91d97d4-7bb2-41be-95b0-d60eb73e58be admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:28:13.288286 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91d97d4-7bb2-41be-95b0-d60eb73e58be admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:28:13.288633 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91d97d4-7bb2-41be-95b0-d60eb73e58be admin admin] PATH_INFO: `/v3/domains` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:28:13.289316 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91d97d4-7bb2-41be-95b0-d60eb73e58be admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:28:13.289625 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91d97d4-7bb2-41be-95b0-d60eb73e58be admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:28:13.292197 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91d97d4-7bb2-41be-95b0-d60eb73e58be admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:28:13.312030 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1382/2742] 10.222.0.55 () {58 vars in 1153 bytes} [Fri Sep 23 15:28:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 15:28:13.313729 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49307894-0b91-4dd5-8267-f9e25299c240 service neutron] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:28:13.327141 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49307894-0b91-4dd5-8267-f9e25299c240 service neutron] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:28:13.332421 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878b397f-28c9-4a8e-821e-60dcb27b7664 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:28:13.333873 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878b397f-28c9-4a8e-821e-60dcb27b7664 admin admin] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:28:13.333873 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878b397f-28c9-4a8e-821e-60dcb27b7664 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:28:13.333873 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878b397f-28c9-4a8e-821e-60dcb27b7664 admin admin] PATH_INFO: `/v3/projects` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:28:13.334440 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878b397f-28c9-4a8e-821e-60dcb27b7664 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:28:13.336637 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878b397f-28c9-4a8e-821e-60dcb27b7664 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:28:13.426109 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1361/2743] 10.222.0.55 () {60 vars in 1153 bytes} [Fri Sep 23 15:28:13 2022] POST /identity/v3/projects => generated 383 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:28:13.427594 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91d97d4-7bb2-41be-95b0-d60eb73e58be admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:28:13.441323 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91d97d4-7bb2-41be-95b0-d60eb73e58be admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:28:13.515897 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071304b9-2bf2-4195-ac89-f0b77a30c5be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:28:13.516535 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071304b9-2bf2-4195-ac89-f0b77a30c5be admin admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:28:13.516813 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071304b9-2bf2-4195-ac89-f0b77a30c5be admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:28:13.517084 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071304b9-2bf2-4195-ac89-f0b77a30c5be admin admin] PATH_INFO: `/v3/users` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:28:13.517703 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071304b9-2bf2-4195-ac89-f0b77a30c5be admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:28:13.519656 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071304b9-2bf2-4195-ac89-f0b77a30c5be admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:28:13.545732 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:28:13.619549 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1383/2744] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Sep 23 15:28:13 2022] POST /identity/v3/users => generated 292 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:28:13.620909 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878b397f-28c9-4a8e-821e-60dcb27b7664 admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:28:13.632961 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878b397f-28c9-4a8e-821e-60dcb27b7664 admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:28:13.637933 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18371596-9890-4673-b96b-c68fa84a2ded admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:28:13.638548 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18371596-9890-4673-b96b-c68fa84a2ded admin admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:28:13.638826 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18371596-9890-4673-b96b-c68fa84a2ded admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:28:13.639098 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18371596-9890-4673-b96b-c68fa84a2ded admin admin] PATH_INFO: `/v3/roles` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:28:13.639738 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18371596-9890-4673-b96b-c68fa84a2ded admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:28:13.641396 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18371596-9890-4673-b96b-c68fa84a2ded admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:28:13.653036 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1362/2745] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Sep 23 15:28:13 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:28:13.657249 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071304b9-2bf2-4195-ac89-f0b77a30c5be admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:28:13.669448 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071304b9-2bf2-4195-ac89-f0b77a30c5be admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:28:13.676150 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece30919-b392-4b2f-9440-cd672c07e3a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:28:13.676150 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece30919-b392-4b2f-9440-cd672c07e3a7 admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:28:13.676150 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece30919-b392-4b2f-9440-cd672c07e3a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:28:13.676150 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece30919-b392-4b2f-9440-cd672c07e3a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:28:13.676709 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece30919-b392-4b2f-9440-cd672c07e3a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:28:13.677994 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece30919-b392-4b2f-9440-cd672c07e3a7 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:28:13.692353 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1384/2746] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Sep 23 15:28:13 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:28:13.693492 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18371596-9890-4673-b96b-c68fa84a2ded admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:28:13.704153 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18371596-9890-4673-b96b-c68fa84a2ded admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:28:13.709455 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3edeb9-c331-48ee-b76b-aff8fa7a0ff4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:28:13.710116 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3edeb9-c331-48ee-b76b-aff8fa7a0ff4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:28:13.710399 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3edeb9-c331-48ee-b76b-aff8fa7a0ff4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:28:13.713062 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3edeb9-c331-48ee-b76b-aff8fa7a0ff4 admin admin] PATH_INFO: `/v3/projects/172d42bd3e754d73b85168a076872835/users/3535c37b720f4d5cb73bbe7b76d92e7d/roles/af257be2dee54d6b942f58650196582c` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:28:13.727264 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3edeb9-c331-48ee-b76b-aff8fa7a0ff4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=172d42bd3e754d73b85168a076872835, user_id=3535c37b720f4d5cb73bbe7b76d92e7d, role_id=af257be2dee54d6b942f58650196582c)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:28:13.730633 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3edeb9-c331-48ee-b76b-aff8fa7a0ff4 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:28:13.838918 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1363/2747] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Sep 23 15:28:13 2022] PUT /identity/v3/projects/172d42bd3e754d73b85168a076872835/users/3535c37b720f4d5cb73bbe7b76d92e7d/roles/af257be2dee54d6b942f58650196582c => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 15:28:13.849793 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891fcdd4-3263-41a6-8e62-a336468faba7 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:28:13.850147 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891fcdd4-3263-41a6-8e62-a336468faba7 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:28:13.850495 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891fcdd4-3263-41a6-8e62-a336468faba7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:28:13.883945 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:28:13.887542 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-891fcdd4-3263-41a6-8e62-a336468faba7 None None] MFA Rules not processed for user `3535c37b720f4d5cb73bbe7b76d92e7d`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:28:13.888800 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-891fcdd4-3263-41a6-8e62-a336468faba7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:28:13.963295 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1385/2748] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:28:13 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:28:13.974853 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3edeb9-c331-48ee-b76b-aff8fa7a0ff4 admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:28:13.976651 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-5c3edeb9-c331-48ee-b76b-aff8fa7a0ff4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:28:14.042798 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3edeb9-c331-48ee-b76b-aff8fa7a0ff4 admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:28:14.047017 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41505640-2070-4b04-9041-2567cc604771 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feca6278be904ef8b0dc580a67e477f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:28:14.047765 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41505640-2070-4b04-9041-2567cc604771 service cinder] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:28:14.048085 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41505640-2070-4b04-9041-2567cc604771 service cinder] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:28:14.048363 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41505640-2070-4b04-9041-2567cc604771 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:28:14.058775 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41505640-2070-4b04-9041-2567cc604771 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:28:14.060278 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41505640-2070-4b04-9041-2567cc604771 service cinder] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:28:14.080002 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1364/2749] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:28:13 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:28:19.468480 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb62cce-b63a-497d-b022-6e71a2444fe8 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:28:19.468480 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb62cce-b63a-497d-b022-6e71a2444fe8 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:28:19.468480 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb62cce-b63a-497d-b022-6e71a2444fe8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:28:19.502576 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-bcb62cce-b63a-497d-b022-6e71a2444fe8 None None] MFA Rules not processed for user `97e06913feae4c23a9abbcd508f818d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:28:19.504087 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-bcb62cce-b63a-497d-b022-6e71a2444fe8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:28:19.584410 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1386/2750] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:28:19 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:28:19.604207 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41505640-2070-4b04-9041-2567cc604771 service cinder] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:28:19.605930 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-41505640-2070-4b04-9041-2567cc604771 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:28:19.728208 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41505640-2070-4b04-9041-2567cc604771 service cinder] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:28:19.734511 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee26a885-c4be-4a14-8f25-e2e6e19ebad0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3c6c7a1325a47be924268e1aa1a7a18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:28:19.735174 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee26a885-c4be-4a14-8f25-e2e6e19ebad0 service nova] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:28:19.735460 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee26a885-c4be-4a14-8f25-e2e6e19ebad0 service nova] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:28:19.735824 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee26a885-c4be-4a14-8f25-e2e6e19ebad0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:28:19.746214 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee26a885-c4be-4a14-8f25-e2e6e19ebad0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:28:19.748053 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee26a885-c4be-4a14-8f25-e2e6e19ebad0 service nova] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:28:19.754166 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:28:19.760247 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1365/2751] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:28:19 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:28:23.098378 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85aab52-05a6-48bf-998b-605a540188f5 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:28:23.098911 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85aab52-05a6-48bf-998b-605a540188f5 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:28:23.098911 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85aab52-05a6-48bf-998b-605a540188f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:28:23.125151 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-f85aab52-05a6-48bf-998b-605a540188f5 None None] MFA Rules not processed for user `97e06913feae4c23a9abbcd508f818d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:28:23.126795 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-f85aab52-05a6-48bf-998b-605a540188f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:28:23.135968 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1387/2752] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:28:23 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:28:24.294079 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee26a885-c4be-4a14-8f25-e2e6e19ebad0 service nova] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:28:24.363579 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee26a885-c4be-4a14-8f25-e2e6e19ebad0 service nova] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:28:24.368384 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c73e119-f78e-400f-b4cc-ff7a54fd2a9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:28:24.369341 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c73e119-f78e-400f-b4cc-ff7a54fd2a9c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:28:24.369618 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c73e119-f78e-400f-b4cc-ff7a54fd2a9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:28:24.369888 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c73e119-f78e-400f-b4cc-ff7a54fd2a9c admin admin] PATH_INFO: `/v3/users/3535c37b720f4d5cb73bbe7b76d92e7d` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:28:24.371086 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c73e119-f78e-400f-b4cc-ff7a54fd2a9c admin admin] RBAC: Authorizing `identity:delete_user(user_id=3535c37b720f4d5cb73bbe7b76d92e7d)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:28:24.373079 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c73e119-f78e-400f-b4cc-ff7a54fd2a9c admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:28:24.595483 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-0c73e119-f78e-400f-b4cc-ff7a54fd2a9c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3535c37b720f4d5cb73bbe7b76d92e7d'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:28:24.611227 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-0c73e119-f78e-400f-b4cc-ff7a54fd2a9c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3535c37b720f4d5cb73bbe7b76d92e7d'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:28:24.622877 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-0c73e119-f78e-400f-b4cc-ff7a54fd2a9c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3535c37b720f4d5cb73bbe7b76d92e7d'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:28:24.711121 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-0c73e119-f78e-400f-b4cc-ff7a54fd2a9c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3535c37b720f4d5cb73bbe7b76d92e7d'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:28:24.711786 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-0c73e119-f78e-400f-b4cc-ff7a54fd2a9c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3535c37b720f4d5cb73bbe7b76d92e7d'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:28:24.718352 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1366/2753] 10.222.0.55 () {58 vars in 1224 bytes} [Fri Sep 23 15:28:24 2022] DELETE /identity/v3/users/3535c37b720f4d5cb73bbe7b76d92e7d => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:28:24.722850 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c220fc-095c-4fa4-b9d6-d0f3903e5a57 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:28:24.723359 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c220fc-095c-4fa4-b9d6-d0f3903e5a57 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:28:24.723480 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c220fc-095c-4fa4-b9d6-d0f3903e5a57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:28:24.743140 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-d9c220fc-095c-4fa4-b9d6-d0f3903e5a57 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:28:24.744741 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-d9c220fc-095c-4fa4-b9d6-d0f3903e5a57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:28:24.806922 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1388/2754] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:28:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:28:24.817077 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c73e119-f78e-400f-b4cc-ff7a54fd2a9c admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:28:24.819158 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-0c73e119-f78e-400f-b4cc-ff7a54fd2a9c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:28:24.885570 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c73e119-f78e-400f-b4cc-ff7a54fd2a9c admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:28:24.889484 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c1c687-2878-40a5-964b-ed9e46b32fa5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd259178570864ffeb936ee10dfec6b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:28:24.890105 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c1c687-2878-40a5-964b-ed9e46b32fa5 service neutron] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:28:24.890418 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c1c687-2878-40a5-964b-ed9e46b32fa5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:28:24.890651 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c1c687-2878-40a5-964b-ed9e46b32fa5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:28:24.901402 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c1c687-2878-40a5-964b-ed9e46b32fa5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:28:24.902889 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c1c687-2878-40a5-964b-ed9e46b32fa5 service neutron] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:28:24.913050 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1367/2755] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:28:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:28:25.706876 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c220fc-095c-4fa4-b9d6-d0f3903e5a57 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:28:25.708339 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-d9c220fc-095c-4fa4-b9d6-d0f3903e5a57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:28:25.722856 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c220fc-095c-4fa4-b9d6-d0f3903e5a57 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:28:25.735168 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228f9f5e-e42e-4201-a819-5394df65ef9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:28:25.736040 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228f9f5e-e42e-4201-a819-5394df65ef9c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:28:25.736539 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228f9f5e-e42e-4201-a819-5394df65ef9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:28:25.736882 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228f9f5e-e42e-4201-a819-5394df65ef9c admin admin] PATH_INFO: `/v3/projects/172d42bd3e754d73b85168a076872835` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:28:25.738359 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228f9f5e-e42e-4201-a819-5394df65ef9c admin admin] RBAC: Authorizing `identity:delete_project(project_id=172d42bd3e754d73b85168a076872835)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:28:25.740865 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228f9f5e-e42e-4201-a819-5394df65ef9c admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:28:25.767216 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:28:25.855506 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-228f9f5e-e42e-4201-a819-5394df65ef9c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '172d42bd3e754d73b85168a076872835', 'request_id': 'req-228f9f5e-e42e-4201-a819-5394df65ef9c'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:28:25.862099 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-228f9f5e-e42e-4201-a819-5394df65ef9c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '172d42bd3e754d73b85168a076872835', 'request_id': 'req-228f9f5e-e42e-4201-a819-5394df65ef9c'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:28:25.869134 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-228f9f5e-e42e-4201-a819-5394df65ef9c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '172d42bd3e754d73b85168a076872835', 'request_id': 'req-228f9f5e-e42e-4201-a819-5394df65ef9c'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:28:25.946247 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-228f9f5e-e42e-4201-a819-5394df65ef9c admin admin] The token cache is being invalidate because project 172d42bd3e754d73b85168a076872835 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81994) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 15:28:25.946656 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-228f9f5e-e42e-4201-a819-5394df65ef9c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:28:25.947367 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-228f9f5e-e42e-4201-a819-5394df65ef9c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:28:25.949408 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1389/2756] 10.222.0.55 () {58 vars in 1233 bytes} [Fri Sep 23 15:28:25 2022] DELETE /identity/v3/projects/172d42bd3e754d73b85168a076872835 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:28:25.954452 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d16766-b336-4038-bdfd-22d944d8ebf2 None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:28:25.955022 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d16766-b336-4038-bdfd-22d944d8ebf2 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:28:25.955022 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d16766-b336-4038-bdfd-22d944d8ebf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:28:25.980903 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-c2d16766-b336-4038-bdfd-22d944d8ebf2 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:28:25.982535 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-c2d16766-b336-4038-bdfd-22d944d8ebf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:28:26.006146 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:28:26.058747 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1368/2757] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:28:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 15:28:26.070444 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7217f91-bf03-4874-a11c-cc6b6ef3cb16 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:28:26.070913 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7217f91-bf03-4874-a11c-cc6b6ef3cb16 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:28:26.072038 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7217f91-bf03-4874-a11c-cc6b6ef3cb16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:28:26.096610 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-f7217f91-bf03-4874-a11c-cc6b6ef3cb16 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:28:26.098726 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-f7217f91-bf03-4874-a11c-cc6b6ef3cb16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:28:26.108536 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1390/2758] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:28:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:28:26.113152 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d16766-b336-4038-bdfd-22d944d8ebf2 None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:28:26.124651 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d16766-b336-4038-bdfd-22d944d8ebf2 None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:28:26.129419 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a98b33-ce46-4687-815b-5575448e97c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:28:26.130089 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a98b33-ce46-4687-815b-5575448e97c2 admin admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:28:26.130372 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a98b33-ce46-4687-815b-5575448e97c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:28:26.130647 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a98b33-ce46-4687-815b-5575448e97c2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:28:26.131290 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a98b33-ce46-4687-815b-5575448e97c2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:28:26.131616 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a98b33-ce46-4687-815b-5575448e97c2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:28:26.133338 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a98b33-ce46-4687-815b-5575448e97c2 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:28:26.149120 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1369/2759] 10.222.0.55 () {58 vars in 1153 bytes} [Fri Sep 23 15:28:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 15:28:26.154153 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7217f91-bf03-4874-a11c-cc6b6ef3cb16 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:28:26.163318 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7217f91-bf03-4874-a11c-cc6b6ef3cb16 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:28:26.168107 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269df451-f7b7-40db-b802-fa36f3c1b61e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:28:26.168702 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269df451-f7b7-40db-b802-fa36f3c1b61e admin admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:28:26.168979 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269df451-f7b7-40db-b802-fa36f3c1b61e admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:28:26.169249 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269df451-f7b7-40db-b802-fa36f3c1b61e admin admin] PATH_INFO: `/v3/projects` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:28:26.169903 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269df451-f7b7-40db-b802-fa36f3c1b61e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:28:26.171770 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269df451-f7b7-40db-b802-fa36f3c1b61e admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:28:26.308842 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1391/2760] 10.222.0.55 () {60 vars in 1153 bytes} [Fri Sep 23 15:28:26 2022] POST /identity/v3/projects => generated 371 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:28:26.313129 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a98b33-ce46-4687-815b-5575448e97c2 admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:28:26.324227 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a98b33-ce46-4687-815b-5575448e97c2 admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:28:26.385268 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5939ae2e-ff65-44b3-b55a-9d644ddae7ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:28:26.385923 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5939ae2e-ff65-44b3-b55a-9d644ddae7ee admin admin] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:28:26.386169 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5939ae2e-ff65-44b3-b55a-9d644ddae7ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:28:26.386446 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5939ae2e-ff65-44b3-b55a-9d644ddae7ee admin admin] PATH_INFO: `/v3/users` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:28:26.387099 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5939ae2e-ff65-44b3-b55a-9d644ddae7ee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:28:26.389067 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5939ae2e-ff65-44b3-b55a-9d644ddae7ee admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:28:26.477561 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1370/2761] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Sep 23 15:28:26 2022] POST /identity/v3/users => generated 286 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 23 15:28:26.482415 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269df451-f7b7-40db-b802-fa36f3c1b61e admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:28:26.493264 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269df451-f7b7-40db-b802-fa36f3c1b61e admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:28:26.498669 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02964d8-3f16-4a28-9b57-91e402738c6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:28:26.499284 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02964d8-3f16-4a28-9b57-91e402738c6f admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:28:26.499573 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02964d8-3f16-4a28-9b57-91e402738c6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:28:26.505755 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02964d8-3f16-4a28-9b57-91e402738c6f admin admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:28:26.505755 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02964d8-3f16-4a28-9b57-91e402738c6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:28:26.505755 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02964d8-3f16-4a28-9b57-91e402738c6f admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:28:26.518646 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1392/2762] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Sep 23 15:28:26 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 23 15:28:26.524554 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5939ae2e-ff65-44b3-b55a-9d644ddae7ee admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:28:26.539152 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5939ae2e-ff65-44b3-b55a-9d644ddae7ee admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:28:26.545840 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5273423f-f2a4-4e24-9ba6-3aaaa4420949 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:28:26.546547 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5273423f-f2a4-4e24-9ba6-3aaaa4420949 admin admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:28:26.546807 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5273423f-f2a4-4e24-9ba6-3aaaa4420949 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:28:26.548067 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5273423f-f2a4-4e24-9ba6-3aaaa4420949 admin admin] PATH_INFO: `/v3/roles` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:28:26.548067 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5273423f-f2a4-4e24-9ba6-3aaaa4420949 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:28:26.549634 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5273423f-f2a4-4e24-9ba6-3aaaa4420949 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:28:26.561906 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1371/2763] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Sep 23 15:28:26 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 23 15:28:26.566118 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02964d8-3f16-4a28-9b57-91e402738c6f admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:28:26.579230 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02964d8-3f16-4a28-9b57-91e402738c6f admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:28:26.584199 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffe2aad-d9e8-4c1f-9421-1740ee6a4d46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:28:26.584927 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffe2aad-d9e8-4c1f-9421-1740ee6a4d46 admin admin] REQUEST_METHOD: `PUT` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:28:26.589782 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffe2aad-d9e8-4c1f-9421-1740ee6a4d46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:28:26.589782 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffe2aad-d9e8-4c1f-9421-1740ee6a4d46 admin admin] PATH_INFO: `/v3/projects/719783ecf917404294351bbb445b9f1b/users/78bb70c49abc42cabecb413112fbb31a/roles/af257be2dee54d6b942f58650196582c` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:28:26.605573 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffe2aad-d9e8-4c1f-9421-1740ee6a4d46 admin admin] RBAC: Authorizing `identity:create_grant(project_id=719783ecf917404294351bbb445b9f1b, user_id=78bb70c49abc42cabecb413112fbb31a, role_id=af257be2dee54d6b942f58650196582c)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:28:26.609113 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffe2aad-d9e8-4c1f-9421-1740ee6a4d46 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:28:26.723004 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1393/2764] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Sep 23 15:28:26 2022] PUT /identity/v3/projects/719783ecf917404294351bbb445b9f1b/users/78bb70c49abc42cabecb413112fbb31a/roles/af257be2dee54d6b942f58650196582c => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:28:26.728987 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f3b943-4834-469a-b4f7-3e77d30abf9b None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:28:26.729206 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f3b943-4834-469a-b4f7-3e77d30abf9b None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:28:26.729495 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f3b943-4834-469a-b4f7-3e77d30abf9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:28:26.756378 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-29f3b943-4834-469a-b4f7-3e77d30abf9b None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:28:26.757876 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-29f3b943-4834-469a-b4f7-3e77d30abf9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:28:26.824964 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1372/2765] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:28:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 15:28:26.835245 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffe2aad-d9e8-4c1f-9421-1740ee6a4d46 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:28:26.842098 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-4ffe2aad-d9e8-4c1f-9421-1740ee6a4d46 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:28:26.899745 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffe2aad-d9e8-4c1f-9421-1740ee6a4d46 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:28:26.903674 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef995a69-18c2-4c8e-9669-b4d7158a650d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd259178570864ffeb936ee10dfec6b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:28:26.904329 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef995a69-18c2-4c8e-9669-b4d7158a650d service neutron] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:28:26.904616 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef995a69-18c2-4c8e-9669-b4d7158a650d service neutron] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:28:26.904890 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef995a69-18c2-4c8e-9669-b4d7158a650d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:28:26.914772 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef995a69-18c2-4c8e-9669-b4d7158a650d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:28:26.916372 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef995a69-18c2-4c8e-9669-b4d7158a650d service neutron] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:28:26.927036 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1394/2766] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:28:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:28:28.867188 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a131e5-0a69-4f5e-bccc-2ee11adc2fa7 None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:28:28.868332 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a131e5-0a69-4f5e-bccc-2ee11adc2fa7 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:28:28.868487 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a131e5-0a69-4f5e-bccc-2ee11adc2fa7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:28:28.942077 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-17a131e5-0a69-4f5e-bccc-2ee11adc2fa7 None None] MFA Rules not processed for user `78bb70c49abc42cabecb413112fbb31a`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:28:28.942077 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-17a131e5-0a69-4f5e-bccc-2ee11adc2fa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:28:28.989602 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:28:29.058249 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1373/2767] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:28:28 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:28:29.069407 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef995a69-18c2-4c8e-9669-b4d7158a650d service neutron] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:28:29.071016 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-ef995a69-18c2-4c8e-9669-b4d7158a650d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:28:29.138330 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:DELETE=7,keystone:SELECT=90,keystone:INSERT=3 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:28:29.144781 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef995a69-18c2-4c8e-9669-b4d7158a650d service neutron] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:28:29.150063 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c0ab83-a301-4297-8f35-0900b9b497ed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feca6278be904ef8b0dc580a67e477f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:28:29.154166 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c0ab83-a301-4297-8f35-0900b9b497ed service cinder] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:28:29.154166 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c0ab83-a301-4297-8f35-0900b9b497ed service cinder] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:28:29.154166 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c0ab83-a301-4297-8f35-0900b9b497ed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:28:29.162949 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c0ab83-a301-4297-8f35-0900b9b497ed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:28:29.165326 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c0ab83-a301-4297-8f35-0900b9b497ed service cinder] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:28:29.178306 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1395/2768] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:28:29 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:28:31.149559 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9283190f-1edf-483c-8da7-a5803d2988d8 None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:28:31.150180 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9283190f-1edf-483c-8da7-a5803d2988d8 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:28:31.150437 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9283190f-1edf-483c-8da7-a5803d2988d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:28:31.172269 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-9283190f-1edf-483c-8da7-a5803d2988d8 None None] MFA Rules not processed for user `78bb70c49abc42cabecb413112fbb31a`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:28:31.173575 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-9283190f-1edf-483c-8da7-a5803d2988d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:28:31.180704 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1374/2769] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:28:31 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:28:39.431096 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=4 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:28:41.174919 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=22 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:28:51.869129 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49286026-fb4b-4573-8e8b-52b65ead99ff None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:28:51.869745 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49286026-fb4b-4573-8e8b-52b65ead99ff None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:28:51.869745 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49286026-fb4b-4573-8e8b-52b65ead99ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:28:51.890574 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-49286026-fb4b-4573-8e8b-52b65ead99ff None None] MFA Rules not processed for user `78bb70c49abc42cabecb413112fbb31a`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:28:51.891883 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-49286026-fb4b-4573-8e8b-52b65ead99ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:28:51.899464 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1396/2770] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:28:51 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:29:01.884685 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=3 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:29:18.207595 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bce7009-48ac-4e79-a524-cadc3c3ee722 None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:29:18.208147 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bce7009-48ac-4e79-a524-cadc3c3ee722 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:29:18.208246 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bce7009-48ac-4e79-a524-cadc3c3ee722 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:29:18.229108 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-7bce7009-48ac-4e79-a524-cadc3c3ee722 None None] MFA Rules not processed for user `68206a6b4f244d45bf7bf1504396e430`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:29:18.232623 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-7bce7009-48ac-4e79-a524-cadc3c3ee722 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:29:18.295180 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1375/2771] 10.222.0.55 () {58 vars in 1009 bytes} [Fri Sep 23 15:29:18 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:29:18.446714 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49286026-fb4b-4573-8e8b-52b65ead99ff None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:29:18.447749 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-49286026-fb4b-4573-8e8b-52b65ead99ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:29:18.520047 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49286026-fb4b-4573-8e8b-52b65ead99ff None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:29:18.525788 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc255b6-5a81-4bfb-a236-2a15b4b2cea4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2228c0b49a0249bbaf524a8c4e9e108b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:29:18.526413 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc255b6-5a81-4bfb-a236-2a15b4b2cea4 service swift] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:29:18.526698 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc255b6-5a81-4bfb-a236-2a15b4b2cea4 service swift] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:29:18.526974 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc255b6-5a81-4bfb-a236-2a15b4b2cea4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:29:18.535948 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc255b6-5a81-4bfb-a236-2a15b4b2cea4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:29:18.537590 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc255b6-5a81-4bfb-a236-2a15b4b2cea4 service swift] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:29:18.547078 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1397/2772] 10.222.0.55 () {60 vars in 1384 bytes} [Fri Sep 23 15:29:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 23 15:29:28.289687 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=25 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:29:28.543137 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=32 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:29:30.079409 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31485638-6321-49ad-a3cf-28b99118a177 None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:29:30.080276 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31485638-6321-49ad-a3cf-28b99118a177 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:29:30.080778 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31485638-6321-49ad-a3cf-28b99118a177 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:29:30.101904 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-31485638-6321-49ad-a3cf-28b99118a177 None None] MFA Rules not processed for user `97e06913feae4c23a9abbcd508f818d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:29:30.103125 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-31485638-6321-49ad-a3cf-28b99118a177 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:29:30.175511 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1376/2773] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:29:30 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:29:36.065088 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475cf7a8-6dc5-4161-b66a-d56c65a7e89d None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:29:36.065911 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475cf7a8-6dc5-4161-b66a-d56c65a7e89d None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:29:36.065911 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475cf7a8-6dc5-4161-b66a-d56c65a7e89d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:29:36.087750 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-475cf7a8-6dc5-4161-b66a-d56c65a7e89d None None] MFA Rules not processed for user `68206a6b4f244d45bf7bf1504396e430`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:29:36.089071 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-475cf7a8-6dc5-4161-b66a-d56c65a7e89d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:29:36.096560 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1398/2774] 10.222.0.55 () {58 vars in 967 bytes} [Fri Sep 23 15:29:36 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:29:36.110048 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31485638-6321-49ad-a3cf-28b99118a177 None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:29:36.123719 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31485638-6321-49ad-a3cf-28b99118a177 None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:29:36.130327 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12b31e9-c30f-463e-af26-023427adce6f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2228c0b49a0249bbaf524a8c4e9e108b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:29:36.130963 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12b31e9-c30f-463e-af26-023427adce6f service swift] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:29:36.131249 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12b31e9-c30f-463e-af26-023427adce6f service swift] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:29:36.131525 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12b31e9-c30f-463e-af26-023427adce6f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:29:36.142537 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12b31e9-c30f-463e-af26-023427adce6f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:29:36.144465 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12b31e9-c30f-463e-af26-023427adce6f service swift] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:29:36.155497 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1377/2775] 10.222.0.55 () {60 vars in 1384 bytes} [Fri Sep 23 15:29:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 23 15:29:39.896111 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475cf7a8-6dc5-4161-b66a-d56c65a7e89d None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:29:39.914663 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475cf7a8-6dc5-4161-b66a-d56c65a7e89d None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:29:39.918551 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373e625b-be05-4efd-928c-fab539a49fd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:29:39.919212 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373e625b-be05-4efd-928c-fab539a49fd5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:29:39.919488 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373e625b-be05-4efd-928c-fab539a49fd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:29:39.919916 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373e625b-be05-4efd-928c-fab539a49fd5 admin admin] PATH_INFO: `/v3/users/78bb70c49abc42cabecb413112fbb31a` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:29:39.921452 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373e625b-be05-4efd-928c-fab539a49fd5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=78bb70c49abc42cabecb413112fbb31a)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:29:39.923371 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373e625b-be05-4efd-928c-fab539a49fd5 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:29:40.314598 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-373e625b-be05-4efd-928c-fab539a49fd5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '78bb70c49abc42cabecb413112fbb31a'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:29:40.329151 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-373e625b-be05-4efd-928c-fab539a49fd5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '78bb70c49abc42cabecb413112fbb31a'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:29:40.337835 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-373e625b-be05-4efd-928c-fab539a49fd5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '78bb70c49abc42cabecb413112fbb31a'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:29:40.474186 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-373e625b-be05-4efd-928c-fab539a49fd5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '78bb70c49abc42cabecb413112fbb31a'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:29:40.475425 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-373e625b-be05-4efd-928c-fab539a49fd5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '78bb70c49abc42cabecb413112fbb31a'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:29:40.481888 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1399/2776] 10.222.0.55 () {58 vars in 1224 bytes} [Fri Sep 23 15:29:39 2022] DELETE /identity/v3/users/78bb70c49abc42cabecb413112fbb31a => generated 0 bytes in 587 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:29:40.832075 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12b31e9-c30f-463e-af26-023427adce6f service swift] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:29:40.872868 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12b31e9-c30f-463e-af26-023427adce6f service swift] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:29:40.915868 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9dbbc2-7102-4286-b3e3-40f10baea8e5 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:29:40.916458 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9dbbc2-7102-4286-b3e3-40f10baea8e5 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:29:40.916458 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9dbbc2-7102-4286-b3e3-40f10baea8e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:29:40.938022 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-ef9dbbc2-7102-4286-b3e3-40f10baea8e5 None None] MFA Rules not processed for user `1cb120cee6bf4ef4b7da15735f5f77e1`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:29:40.939280 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-ef9dbbc2-7102-4286-b3e3-40f10baea8e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:29:40.959960 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73dcffd-d21d-4073-bab2-17d463fd73ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:29:40.960681 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73dcffd-d21d-4073-bab2-17d463fd73ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:29:40.965555 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73dcffd-d21d-4073-bab2-17d463fd73ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:29:40.965555 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73dcffd-d21d-4073-bab2-17d463fd73ec admin admin] PATH_INFO: `/v3/projects/719783ecf917404294351bbb445b9f1b` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:29:40.965555 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73dcffd-d21d-4073-bab2-17d463fd73ec admin admin] RBAC: Authorizing `identity:delete_project(project_id=719783ecf917404294351bbb445b9f1b)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:29:40.965555 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73dcffd-d21d-4073-bab2-17d463fd73ec admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:29:41.004670 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1400/2777] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:29:40 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 15:29:41.098931 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-b73dcffd-d21d-4073-bab2-17d463fd73ec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '719783ecf917404294351bbb445b9f1b', 'request_id': 'req-b73dcffd-d21d-4073-bab2-17d463fd73ec'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:29:41.105464 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-b73dcffd-d21d-4073-bab2-17d463fd73ec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '719783ecf917404294351bbb445b9f1b', 'request_id': 'req-b73dcffd-d21d-4073-bab2-17d463fd73ec'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:29:41.212514 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-b73dcffd-d21d-4073-bab2-17d463fd73ec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '719783ecf917404294351bbb445b9f1b', 'request_id': 'req-b73dcffd-d21d-4073-bab2-17d463fd73ec'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:29:41.228512 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-b73dcffd-d21d-4073-bab2-17d463fd73ec admin admin] The token cache is being invalidate because project 719783ecf917404294351bbb445b9f1b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81993) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 15:29:41.229100 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-b73dcffd-d21d-4073-bab2-17d463fd73ec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:29:41.230038 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-b73dcffd-d21d-4073-bab2-17d463fd73ec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:29:41.231867 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1378/2778] 10.222.0.55 () {58 vars in 1233 bytes} [Fri Sep 23 15:29:40 2022] DELETE /identity/v3/projects/719783ecf917404294351bbb445b9f1b => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:29:50.997291 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=57,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:29:51.222519 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=82,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:30:24.636566 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9dbbc2-7102-4286-b3e3-40f10baea8e5 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:30:24.638735 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-ef9dbbc2-7102-4286-b3e3-40f10baea8e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:30:24.706183 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9dbbc2-7102-4286-b3e3-40f10baea8e5 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:30:24.759337 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac38977-59fa-4b84-b4c4-289500d98703 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feca6278be904ef8b0dc580a67e477f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:30:24.759957 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac38977-59fa-4b84-b4c4-289500d98703 service cinder] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:30:24.760267 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac38977-59fa-4b84-b4c4-289500d98703 service cinder] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:30:24.760541 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac38977-59fa-4b84-b4c4-289500d98703 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:30:24.762312 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-2ac38977-59fa-4b84-b4c4-289500d98703 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:30:24.842388 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac38977-59fa-4b84-b4c4-289500d98703 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:30:24.843745 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac38977-59fa-4b84-b4c4-289500d98703 service cinder] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:30:24.857526 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1401/2779] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:30:24 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:30:34.850580 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=74 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:30:42.821385 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc3e313-060e-482a-8e57-6fd5ee963d66 None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:30:42.821982 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc3e313-060e-482a-8e57-6fd5ee963d66 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:30:42.822202 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc3e313-060e-482a-8e57-6fd5ee963d66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:30:42.850050 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-8fc3e313-060e-482a-8e57-6fd5ee963d66 None None] MFA Rules not processed for user `97e06913feae4c23a9abbcd508f818d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:30:42.851295 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-8fc3e313-060e-482a-8e57-6fd5ee963d66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:30:42.914371 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1379/2780] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:30:42 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:30:43.767377 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac38977-59fa-4b84-b4c4-289500d98703 service cinder] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:30:43.770161 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-2ac38977-59fa-4b84-b4c4-289500d98703 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:30:43.912087 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac38977-59fa-4b84-b4c4-289500d98703 service cinder] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:30:43.912087 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0a9ebb-f4a4-408d-bb29-375bf4ba01ec service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09f1a55d180642c781a6e0b017d9f0aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:30:43.912704 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0a9ebb-f4a4-408d-bb29-375bf4ba01ec service placement] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:30:43.912704 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0a9ebb-f4a4-408d-bb29-375bf4ba01ec service placement] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:30:43.913343 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0a9ebb-f4a4-408d-bb29-375bf4ba01ec service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:30:43.916203 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-5c0a9ebb-f4a4-408d-bb29-375bf4ba01ec service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:30:43.931574 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0a9ebb-f4a4-408d-bb29-375bf4ba01ec service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:30:43.933296 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0a9ebb-f4a4-408d-bb29-375bf4ba01ec service placement] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:30:43.947395 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1402/2781] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:30:43 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:30:48.220892 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc3e313-060e-482a-8e57-6fd5ee963d66 None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:30:48.229273 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-8fc3e313-060e-482a-8e57-6fd5ee963d66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:30:48.292237 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc3e313-060e-482a-8e57-6fd5ee963d66 None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:30:48.296010 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00d1221-a54f-4f65-81d9-9145ae18cff2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd259178570864ffeb936ee10dfec6b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:30:48.298004 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00d1221-a54f-4f65-81d9-9145ae18cff2 service neutron] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:30:48.298004 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00d1221-a54f-4f65-81d9-9145ae18cff2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:30:48.298004 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00d1221-a54f-4f65-81d9-9145ae18cff2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:30:48.300797 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-f00d1221-a54f-4f65-81d9-9145ae18cff2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:30:48.313187 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00d1221-a54f-4f65-81d9-9145ae18cff2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:30:48.314685 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00d1221-a54f-4f65-81d9-9145ae18cff2 service neutron] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:30:48.324521 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1380/2782] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:30:48 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:30:49.665088 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0a9ebb-f4a4-408d-bb29-375bf4ba01ec service placement] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:30:49.665088 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-5c0a9ebb-f4a4-408d-bb29-375bf4ba01ec service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:30:49.665088 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0a9ebb-f4a4-408d-bb29-375bf4ba01ec service placement] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:30:49.665836 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d9acc2-d991-4a31-9786-975af13be27a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3c6c7a1325a47be924268e1aa1a7a18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:30:49.665836 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d9acc2-d991-4a31-9786-975af13be27a service nova] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:30:49.665836 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d9acc2-d991-4a31-9786-975af13be27a service nova] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:30:49.665836 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d9acc2-d991-4a31-9786-975af13be27a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:30:49.666248 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-23d9acc2-d991-4a31-9786-975af13be27a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:30:49.681571 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d9acc2-d991-4a31-9786-975af13be27a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:30:49.683194 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d9acc2-d991-4a31-9786-975af13be27a service nova] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:30:49.695825 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1403/2783] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:30:49 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:30:54.381719 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00d1221-a54f-4f65-81d9-9145ae18cff2 service neutron] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:30:54.396577 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00d1221-a54f-4f65-81d9-9145ae18cff2 service neutron] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:30:54.402523 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1dfc0b2-a6e9-4a54-ae3e-cbb072cb42ab service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09f1a55d180642c781a6e0b017d9f0aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:30:54.403155 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dfc0b2-a6e9-4a54-ae3e-cbb072cb42ab service placement] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:30:54.403495 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dfc0b2-a6e9-4a54-ae3e-cbb072cb42ab service placement] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:30:54.403777 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dfc0b2-a6e9-4a54-ae3e-cbb072cb42ab service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:30:54.405945 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-f1dfc0b2-a6e9-4a54-ae3e-cbb072cb42ab service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:30:54.420554 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1dfc0b2-a6e9-4a54-ae3e-cbb072cb42ab service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:30:54.428103 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1dfc0b2-a6e9-4a54-ae3e-cbb072cb42ab service placement] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:30:54.434212 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1381/2784] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:30:54 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:30:54.512984 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d9acc2-d991-4a31-9786-975af13be27a service nova] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:30:54.514630 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-23d9acc2-d991-4a31-9786-975af13be27a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:30:54.586156 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d9acc2-d991-4a31-9786-975af13be27a service nova] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:30:54.590665 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270df1b8-06a9-4bdd-ae61-244f10ebcc42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:30:54.591414 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270df1b8-06a9-4bdd-ae61-244f10ebcc42 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:30:54.591608 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270df1b8-06a9-4bdd-ae61-244f10ebcc42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:30:54.591932 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270df1b8-06a9-4bdd-ae61-244f10ebcc42 admin admin] PATH_INFO: `/v3/users/1cb120cee6bf4ef4b7da15735f5f77e1` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:30:54.593055 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270df1b8-06a9-4bdd-ae61-244f10ebcc42 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1cb120cee6bf4ef4b7da15735f5f77e1)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:30:54.594951 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270df1b8-06a9-4bdd-ae61-244f10ebcc42 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:30:54.917703 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-270df1b8-06a9-4bdd-ae61-244f10ebcc42 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1cb120cee6bf4ef4b7da15735f5f77e1'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:30:54.932398 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-270df1b8-06a9-4bdd-ae61-244f10ebcc42 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1cb120cee6bf4ef4b7da15735f5f77e1'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:30:54.940712 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-270df1b8-06a9-4bdd-ae61-244f10ebcc42 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1cb120cee6bf4ef4b7da15735f5f77e1'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:30:54.947467 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:30:55.052183 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-270df1b8-06a9-4bdd-ae61-244f10ebcc42 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1cb120cee6bf4ef4b7da15735f5f77e1'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:30:55.052825 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-270df1b8-06a9-4bdd-ae61-244f10ebcc42 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1cb120cee6bf4ef4b7da15735f5f77e1'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:30:55.059566 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1404/2785] 10.222.0.55 () {58 vars in 1224 bytes} [Fri Sep 23 15:30:54 2022] DELETE /identity/v3/users/1cb120cee6bf4ef4b7da15735f5f77e1 => generated 0 bytes in 547 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:30:55.361856 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1dfc0b2-a6e9-4a54-ae3e-cbb072cb42ab service placement] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:30:55.380095 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1dfc0b2-a6e9-4a54-ae3e-cbb072cb42ab service placement] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:30:55.435396 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66fddc6-230b-48e7-92e0-27c225891048 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:30:55.436125 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66fddc6-230b-48e7-92e0-27c225891048 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:30:55.436337 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66fddc6-230b-48e7-92e0-27c225891048 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:30:55.436613 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66fddc6-230b-48e7-92e0-27c225891048 admin admin] PATH_INFO: `/v3/projects/d86aa99c129b476980f17821cd30ef30` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:30:55.437751 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66fddc6-230b-48e7-92e0-27c225891048 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d86aa99c129b476980f17821cd30ef30)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:30:55.439757 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66fddc6-230b-48e7-92e0-27c225891048 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:30:55.578783 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:30:55.587827 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-c66fddc6-230b-48e7-92e0-27c225891048 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd86aa99c129b476980f17821cd30ef30', 'request_id': 'req-c66fddc6-230b-48e7-92e0-27c225891048'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:30:55.594071 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-c66fddc6-230b-48e7-92e0-27c225891048 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd86aa99c129b476980f17821cd30ef30', 'request_id': 'req-c66fddc6-230b-48e7-92e0-27c225891048'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:30:55.677601 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-c66fddc6-230b-48e7-92e0-27c225891048 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd86aa99c129b476980f17821cd30ef30', 'request_id': 'req-c66fddc6-230b-48e7-92e0-27c225891048'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:30:55.689461 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-c66fddc6-230b-48e7-92e0-27c225891048 admin admin] The token cache is being invalidate because project d86aa99c129b476980f17821cd30ef30 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81993) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 15:30:55.689747 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-c66fddc6-230b-48e7-92e0-27c225891048 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:30:55.690443 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-c66fddc6-230b-48e7-92e0-27c225891048 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:30:55.692090 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1382/2786] 10.222.0.55 () {58 vars in 1233 bytes} [Fri Sep 23 15:30:55 2022] DELETE /identity/v3/projects/d86aa99c129b476980f17821cd30ef30 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:30:55.696343 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270df1b8-06a9-4bdd-ae61-244f10ebcc42 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:30:55.697798 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-270df1b8-06a9-4bdd-ae61-244f10ebcc42 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:30:55.712524 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270df1b8-06a9-4bdd-ae61-244f10ebcc42 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:30:55.770160 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadb6001-af33-4d56-9e48-b544a92747ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:30:55.770813 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadb6001-af33-4d56-9e48-b544a92747ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:30:55.771099 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadb6001-af33-4d56-9e48-b544a92747ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:30:55.771374 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadb6001-af33-4d56-9e48-b544a92747ff admin admin] PATH_INFO: `/v3/users/b401a9f68da44913aef7366b1e69af65` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:30:55.772593 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dadb6001-af33-4d56-9e48-b544a92747ff admin admin] RBAC: Authorizing `identity:delete_user(user_id=b401a9f68da44913aef7366b1e69af65)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:30:55.774539 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dadb6001-af33-4d56-9e48-b544a92747ff admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:30:55.989798 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-dadb6001-af33-4d56-9e48-b544a92747ff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b401a9f68da44913aef7366b1e69af65'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:30:56.005824 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-dadb6001-af33-4d56-9e48-b544a92747ff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b401a9f68da44913aef7366b1e69af65'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:30:56.014181 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-dadb6001-af33-4d56-9e48-b544a92747ff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b401a9f68da44913aef7366b1e69af65'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:30:56.125439 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-dadb6001-af33-4d56-9e48-b544a92747ff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b401a9f68da44913aef7366b1e69af65'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:30:56.126097 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-dadb6001-af33-4d56-9e48-b544a92747ff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b401a9f68da44913aef7366b1e69af65'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:30:56.133453 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1405/2787] 10.222.0.55 () {58 vars in 1224 bytes} [Fri Sep 23 15:30:55 2022] DELETE /identity/v3/users/b401a9f68da44913aef7366b1e69af65 => generated 0 bytes in 438 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:30:56.396068 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66fddc6-230b-48e7-92e0-27c225891048 admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:30:56.398052 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-c66fddc6-230b-48e7-92e0-27c225891048 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:30:56.412395 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66fddc6-230b-48e7-92e0-27c225891048 admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:30:56.491360 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdda93d1-c94c-4c88-a381-9ca6478b583c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:30:56.492233 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdda93d1-c94c-4c88-a381-9ca6478b583c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:30:56.492627 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdda93d1-c94c-4c88-a381-9ca6478b583c admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:30:56.492902 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdda93d1-c94c-4c88-a381-9ca6478b583c admin admin] PATH_INFO: `/v3/projects/50f11b8c3f7949debec23f52ea06b70c` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:30:56.494082 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdda93d1-c94c-4c88-a381-9ca6478b583c admin admin] RBAC: Authorizing `identity:delete_project(project_id=50f11b8c3f7949debec23f52ea06b70c)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:30:56.496116 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdda93d1-c94c-4c88-a381-9ca6478b583c admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:30:56.694449 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-fdda93d1-c94c-4c88-a381-9ca6478b583c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '50f11b8c3f7949debec23f52ea06b70c', 'request_id': 'req-fdda93d1-c94c-4c88-a381-9ca6478b583c'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:30:56.700911 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-fdda93d1-c94c-4c88-a381-9ca6478b583c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '50f11b8c3f7949debec23f52ea06b70c', 'request_id': 'req-fdda93d1-c94c-4c88-a381-9ca6478b583c'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:30:56.846776 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-fdda93d1-c94c-4c88-a381-9ca6478b583c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '50f11b8c3f7949debec23f52ea06b70c', 'request_id': 'req-fdda93d1-c94c-4c88-a381-9ca6478b583c'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:30:56.859843 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-fdda93d1-c94c-4c88-a381-9ca6478b583c admin admin] The token cache is being invalidate because project 50f11b8c3f7949debec23f52ea06b70c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81993) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 15:30:56.860164 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-fdda93d1-c94c-4c88-a381-9ca6478b583c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:30:56.860845 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-fdda93d1-c94c-4c88-a381-9ca6478b583c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:30:56.862333 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1383/2788] 10.222.0.55 () {58 vars in 1233 bytes} [Fri Sep 23 15:30:56 2022] DELETE /identity/v3/projects/50f11b8c3f7949debec23f52ea06b70c => generated 0 bytes in 467 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:31:06.128985 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=55,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:31:06.856430 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=62,keystone:DELETE=11,keystone:INSERT=2 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:31:52.807676 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e09421-8b64-47d5-a9e4-2020486df2d9 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:31:52.808558 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e09421-8b64-47d5-a9e4-2020486df2d9 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:31:52.808558 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e09421-8b64-47d5-a9e4-2020486df2d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:31:52.828204 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-e7e09421-8b64-47d5-a9e4-2020486df2d9 None None] MFA Rules not processed for user `97e06913feae4c23a9abbcd508f818d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:31:52.829454 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-e7e09421-8b64-47d5-a9e4-2020486df2d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:31:52.885400 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1406/2789] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:31:52 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:32:02.878231 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=25 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:32:04.746543 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdda93d1-c94c-4c88-a381-9ca6478b583c admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:32:04.748612 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-fdda93d1-c94c-4c88-a381-9ca6478b583c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:32:04.826997 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdda93d1-c94c-4c88-a381-9ca6478b583c admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:32:04.833960 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77d9b4e-ec5f-4273-a554-dca2f31cfec2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3c6c7a1325a47be924268e1aa1a7a18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:32:04.834733 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77d9b4e-ec5f-4273-a554-dca2f31cfec2 service nova] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:32:04.835090 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77d9b4e-ec5f-4273-a554-dca2f31cfec2 service nova] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:32:04.835389 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77d9b4e-ec5f-4273-a554-dca2f31cfec2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:32:04.837370 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-c77d9b4e-ec5f-4273-a554-dca2f31cfec2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:32:04.852064 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77d9b4e-ec5f-4273-a554-dca2f31cfec2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:32:04.853754 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77d9b4e-ec5f-4273-a554-dca2f31cfec2 service nova] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:32:04.865843 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1384/2790] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:32:04 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:32:14.860065 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=34 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:33:08.904149 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e09421-8b64-47d5-a9e4-2020486df2d9 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:33:08.905656 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-e7e09421-8b64-47d5-a9e4-2020486df2d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:33:08.974160 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e09421-8b64-47d5-a9e4-2020486df2d9 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:33:08.979390 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73cfd94-fb27-4f65-abbe-1fcdbe4441f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:33:08.980067 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73cfd94-fb27-4f65-abbe-1fcdbe4441f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:33:08.980353 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73cfd94-fb27-4f65-abbe-1fcdbe4441f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:33:08.980629 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73cfd94-fb27-4f65-abbe-1fcdbe4441f0 admin admin] PATH_INFO: `/v3/users/97e06913feae4c23a9abbcd508f818d3` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:33:08.981766 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73cfd94-fb27-4f65-abbe-1fcdbe4441f0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=97e06913feae4c23a9abbcd508f818d3)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:33:08.983685 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73cfd94-fb27-4f65-abbe-1fcdbe4441f0 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:33:09.180498 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-a73cfd94-fb27-4f65-abbe-1fcdbe4441f0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '97e06913feae4c23a9abbcd508f818d3'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:33:09.193575 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-a73cfd94-fb27-4f65-abbe-1fcdbe4441f0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '97e06913feae4c23a9abbcd508f818d3'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:33:09.200836 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-a73cfd94-fb27-4f65-abbe-1fcdbe4441f0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '97e06913feae4c23a9abbcd508f818d3'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:33:09.304693 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-a73cfd94-fb27-4f65-abbe-1fcdbe4441f0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '97e06913feae4c23a9abbcd508f818d3'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:33:09.304693 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-a73cfd94-fb27-4f65-abbe-1fcdbe4441f0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '97e06913feae4c23a9abbcd508f818d3'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:33:09.311172 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1407/2791] 10.222.0.55 () {58 vars in 1224 bytes} [Fri Sep 23 15:33:08 2022] DELETE /identity/v3/users/97e06913feae4c23a9abbcd508f818d3 => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:33:09.568064 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77d9b4e-ec5f-4273-a554-dca2f31cfec2 service nova] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:33:09.570068 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-c77d9b4e-ec5f-4273-a554-dca2f31cfec2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:33:09.595612 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73cfd94-fb27-4f65-abbe-1fcdbe4441f0 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:33:09.597198 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-a73cfd94-fb27-4f65-abbe-1fcdbe4441f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:33:09.649960 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77d9b4e-ec5f-4273-a554-dca2f31cfec2 service nova] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:33:09.679894 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73cfd94-fb27-4f65-abbe-1fcdbe4441f0 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:33:09.684773 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ecde38-3bc1-45e4-86b6-ff32e334e2a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:33:09.685416 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ecde38-3bc1-45e4-86b6-ff32e334e2a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:33:09.685697 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ecde38-3bc1-45e4-86b6-ff32e334e2a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:33:09.685973 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ecde38-3bc1-45e4-86b6-ff32e334e2a8 admin admin] PATH_INFO: `/v3/projects/695b6a2a390b481bbca2fd367273c268` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:33:09.687111 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ecde38-3bc1-45e4-86b6-ff32e334e2a8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=695b6a2a390b481bbca2fd367273c268)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:33:09.689145 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ecde38-3bc1-45e4-86b6-ff32e334e2a8 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:33:09.714369 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:33:09.716940 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d38289-f25d-4990-8c1e-f65ce53cda75 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09f1a55d180642c781a6e0b017d9f0aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:33:09.717542 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d38289-f25d-4990-8c1e-f65ce53cda75 service placement] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:33:09.717824 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d38289-f25d-4990-8c1e-f65ce53cda75 service placement] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:33:09.718141 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d38289-f25d-4990-8c1e-f65ce53cda75 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:33:09.719895 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-e1d38289-f25d-4990-8c1e-f65ce53cda75 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:33:09.732901 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d38289-f25d-4990-8c1e-f65ce53cda75 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:33:09.734505 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d38289-f25d-4990-8c1e-f65ce53cda75 service placement] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:33:09.746578 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1385/2792] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:33:09 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:33:09.844102 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-34ecde38-3bc1-45e4-86b6-ff32e334e2a8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '695b6a2a390b481bbca2fd367273c268', 'request_id': 'req-34ecde38-3bc1-45e4-86b6-ff32e334e2a8'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:33:09.852111 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-34ecde38-3bc1-45e4-86b6-ff32e334e2a8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '695b6a2a390b481bbca2fd367273c268', 'request_id': 'req-34ecde38-3bc1-45e4-86b6-ff32e334e2a8'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:33:09.857888 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-34ecde38-3bc1-45e4-86b6-ff32e334e2a8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '695b6a2a390b481bbca2fd367273c268', 'request_id': 'req-34ecde38-3bc1-45e4-86b6-ff32e334e2a8'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:33:09.919554 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-34ecde38-3bc1-45e4-86b6-ff32e334e2a8 admin admin] The token cache is being invalidate because project 695b6a2a390b481bbca2fd367273c268 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81994) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 15:33:09.920267 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-34ecde38-3bc1-45e4-86b6-ff32e334e2a8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:33:09.920506 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-34ecde38-3bc1-45e4-86b6-ff32e334e2a8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:33:09.922096 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1408/2793] 10.222.0.55 () {58 vars in 1233 bytes} [Fri Sep 23 15:33:09 2022] DELETE /identity/v3/projects/695b6a2a390b481bbca2fd367273c268 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 15:33:09.929206 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c04230f-f42c-426d-91ab-d776219a4a63 None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:33:09.929578 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c04230f-f42c-426d-91ab-d776219a4a63 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:33:09.929795 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c04230f-f42c-426d-91ab-d776219a4a63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:33:09.949950 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-6c04230f-f42c-426d-91ab-d776219a4a63 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:33:09.951281 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-6c04230f-f42c-426d-91ab-d776219a4a63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:33:10.017298 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1386/2794] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:33:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 15:33:10.026410 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b962743-8863-43ee-8e5d-6f10a52d8370 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:33:10.026674 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b962743-8863-43ee-8e5d-6f10a52d8370 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:33:10.026949 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b962743-8863-43ee-8e5d-6f10a52d8370 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:33:10.048391 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-4b962743-8863-43ee-8e5d-6f10a52d8370 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:33:10.049612 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-4b962743-8863-43ee-8e5d-6f10a52d8370 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:33:10.058296 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1409/2795] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:33:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:33:10.062634 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c04230f-f42c-426d-91ab-d776219a4a63 None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:33:10.073225 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c04230f-f42c-426d-91ab-d776219a4a63 None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:33:10.077785 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e7e8fa-ac4c-4e5a-be6f-84050ca2ca2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:33:10.078392 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e7e8fa-ac4c-4e5a-be6f-84050ca2ca2b admin admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:33:10.078668 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e7e8fa-ac4c-4e5a-be6f-84050ca2ca2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:33:10.078938 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e7e8fa-ac4c-4e5a-be6f-84050ca2ca2b admin admin] PATH_INFO: `/v3/domains` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:33:10.079605 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e7e8fa-ac4c-4e5a-be6f-84050ca2ca2b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:33:10.079903 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e7e8fa-ac4c-4e5a-be6f-84050ca2ca2b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:33:10.081547 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e7e8fa-ac4c-4e5a-be6f-84050ca2ca2b admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:33:10.093794 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1387/2796] 10.222.0.55 () {58 vars in 1153 bytes} [Fri Sep 23 15:33:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 15:33:10.098233 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b962743-8863-43ee-8e5d-6f10a52d8370 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:33:10.107639 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b962743-8863-43ee-8e5d-6f10a52d8370 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:33:10.112719 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe46a9c6-b601-4d7c-8ddd-6f573d055b70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:33:10.120736 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe46a9c6-b601-4d7c-8ddd-6f573d055b70 admin admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:33:10.120736 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe46a9c6-b601-4d7c-8ddd-6f573d055b70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:33:10.120736 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe46a9c6-b601-4d7c-8ddd-6f573d055b70 admin admin] PATH_INFO: `/v3/projects` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:33:10.120736 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe46a9c6-b601-4d7c-8ddd-6f573d055b70 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:33:10.120736 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe46a9c6-b601-4d7c-8ddd-6f573d055b70 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:33:10.225206 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1410/2797] 10.222.0.55 () {60 vars in 1153 bytes} [Fri Sep 23 15:33:10 2022] POST /identity/v3/projects => generated 381 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:33:10.229506 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e7e8fa-ac4c-4e5a-be6f-84050ca2ca2b admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:33:10.241750 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e7e8fa-ac4c-4e5a-be6f-84050ca2ca2b admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:33:10.263954 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=100 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:33:10.307103 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c465b0-a0e6-4fd4-8bdc-fb46cc8157b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:33:10.307711 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c465b0-a0e6-4fd4-8bdc-fb46cc8157b0 admin admin] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:33:10.308003 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c465b0-a0e6-4fd4-8bdc-fb46cc8157b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:33:10.308304 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c465b0-a0e6-4fd4-8bdc-fb46cc8157b0 admin admin] PATH_INFO: `/v3/users` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:33:10.308945 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c465b0-a0e6-4fd4-8bdc-fb46cc8157b0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:33:10.310857 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c465b0-a0e6-4fd4-8bdc-fb46cc8157b0 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:33:10.424017 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1388/2798] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Sep 23 15:33:10 2022] POST /identity/v3/users => generated 291 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:33:10.429163 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe46a9c6-b601-4d7c-8ddd-6f573d055b70 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:33:10.439637 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe46a9c6-b601-4d7c-8ddd-6f573d055b70 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:33:10.445350 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b3fe19-7620-47f1-aa80-9e608fbb4bf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:33:10.445955 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b3fe19-7620-47f1-aa80-9e608fbb4bf8 admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:33:10.446237 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b3fe19-7620-47f1-aa80-9e608fbb4bf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:33:10.446512 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b3fe19-7620-47f1-aa80-9e608fbb4bf8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:33:10.447194 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b3fe19-7620-47f1-aa80-9e608fbb4bf8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:33:10.448809 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b3fe19-7620-47f1-aa80-9e608fbb4bf8 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:33:10.459651 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1411/2799] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Sep 23 15:33:10 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:33:10.463684 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c465b0-a0e6-4fd4-8bdc-fb46cc8157b0 admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:33:10.474624 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c465b0-a0e6-4fd4-8bdc-fb46cc8157b0 admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:33:10.479757 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e247e8c2-9594-47c0-8e74-c66bf1199818 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:33:10.480429 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e247e8c2-9594-47c0-8e74-c66bf1199818 admin admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:33:10.480741 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e247e8c2-9594-47c0-8e74-c66bf1199818 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:33:10.481018 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e247e8c2-9594-47c0-8e74-c66bf1199818 admin admin] PATH_INFO: `/v3/roles` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:33:10.481667 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e247e8c2-9594-47c0-8e74-c66bf1199818 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:33:10.483290 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e247e8c2-9594-47c0-8e74-c66bf1199818 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:33:10.496391 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1389/2800] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Sep 23 15:33:10 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:33:10.500763 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b3fe19-7620-47f1-aa80-9e608fbb4bf8 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:33:10.512896 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b3fe19-7620-47f1-aa80-9e608fbb4bf8 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:33:10.517495 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cc0c30-326a-4f7d-997b-7d1bfe4b8bcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:33:10.518153 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cc0c30-326a-4f7d-997b-7d1bfe4b8bcf admin admin] REQUEST_METHOD: `PUT` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:33:10.518436 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cc0c30-326a-4f7d-997b-7d1bfe4b8bcf admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:33:10.518712 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cc0c30-326a-4f7d-997b-7d1bfe4b8bcf admin admin] PATH_INFO: `/v3/projects/58ccddd1f0c549a2acc867aad5423f9f/users/1279a2e4dd3d4de1af379a2fd6a6078b/roles/af257be2dee54d6b942f58650196582c` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:33:10.535764 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cc0c30-326a-4f7d-997b-7d1bfe4b8bcf admin admin] RBAC: Authorizing `identity:create_grant(project_id=58ccddd1f0c549a2acc867aad5423f9f, user_id=1279a2e4dd3d4de1af379a2fd6a6078b, role_id=af257be2dee54d6b942f58650196582c)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:33:10.539220 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cc0c30-326a-4f7d-997b-7d1bfe4b8bcf admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:33:10.611315 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1412/2801] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Sep 23 15:33:10 2022] PUT /identity/v3/projects/58ccddd1f0c549a2acc867aad5423f9f/users/1279a2e4dd3d4de1af379a2fd6a6078b/roles/af257be2dee54d6b942f58650196582c => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:33:10.623417 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63e5850-a73d-485d-b2d9-7e8d6b4c5fdc None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:33:10.624158 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63e5850-a73d-485d-b2d9-7e8d6b4c5fdc None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:33:10.624158 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63e5850-a73d-485d-b2d9-7e8d6b4c5fdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:33:10.651797 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-a63e5850-a73d-485d-b2d9-7e8d6b4c5fdc None None] MFA Rules not processed for user `1279a2e4dd3d4de1af379a2fd6a6078b`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:33:10.653048 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-a63e5850-a73d-485d-b2d9-7e8d6b4c5fdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:33:10.718173 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1390/2802] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:33:10 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:33:10.722785 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cc0c30-326a-4f7d-997b-7d1bfe4b8bcf admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:33:10.787079 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cc0c30-326a-4f7d-997b-7d1bfe4b8bcf admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:33:10.791538 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36499bb2-5afd-44c5-a59a-75da7549fa9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:33:10.792175 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36499bb2-5afd-44c5-a59a-75da7549fa9a admin admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:33:10.792477 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36499bb2-5afd-44c5-a59a-75da7549fa9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:33:10.792841 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36499bb2-5afd-44c5-a59a-75da7549fa9a admin admin] PATH_INFO: `/v3/projects` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:33:10.793467 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36499bb2-5afd-44c5-a59a-75da7549fa9a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:33:10.795338 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36499bb2-5afd-44c5-a59a-75da7549fa9a admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:33:10.870444 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1413/2803] 10.222.0.55 () {60 vars in 1153 bytes} [Fri Sep 23 15:33:10 2022] POST /identity/v3/projects => generated 381 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:33:10.874735 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63e5850-a73d-485d-b2d9-7e8d6b4c5fdc None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:33:10.885691 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63e5850-a73d-485d-b2d9-7e8d6b4c5fdc None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:33:10.944338 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370ce9d2-384b-48f2-80e1-88a6029c5b77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:33:10.944959 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370ce9d2-384b-48f2-80e1-88a6029c5b77 admin admin] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:33:10.945230 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370ce9d2-384b-48f2-80e1-88a6029c5b77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:33:10.945494 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370ce9d2-384b-48f2-80e1-88a6029c5b77 admin admin] PATH_INFO: `/v3/users` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:33:10.946127 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370ce9d2-384b-48f2-80e1-88a6029c5b77 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:33:10.948061 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370ce9d2-384b-48f2-80e1-88a6029c5b77 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:33:11.023287 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1391/2804] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Sep 23 15:33:10 2022] POST /identity/v3/users => generated 297 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:33:11.027395 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36499bb2-5afd-44c5-a59a-75da7549fa9a admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:33:11.039047 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36499bb2-5afd-44c5-a59a-75da7549fa9a admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:33:11.043627 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa78ad32-87bf-45c8-a0c3-ec62007f03d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:33:11.044305 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa78ad32-87bf-45c8-a0c3-ec62007f03d9 admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:33:11.044593 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa78ad32-87bf-45c8-a0c3-ec62007f03d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:33:11.044878 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa78ad32-87bf-45c8-a0c3-ec62007f03d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:33:11.045521 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa78ad32-87bf-45c8-a0c3-ec62007f03d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:33:11.047104 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa78ad32-87bf-45c8-a0c3-ec62007f03d9 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:33:11.057950 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1414/2805] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Sep 23 15:33:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:33:11.062411 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370ce9d2-384b-48f2-80e1-88a6029c5b77 admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:33:11.073389 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370ce9d2-384b-48f2-80e1-88a6029c5b77 admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:33:11.078020 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6506e64-4bdc-4f8a-afd2-c8dc35c8f2e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:33:11.078679 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6506e64-4bdc-4f8a-afd2-c8dc35c8f2e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:33:11.078959 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6506e64-4bdc-4f8a-afd2-c8dc35c8f2e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:33:11.079236 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6506e64-4bdc-4f8a-afd2-c8dc35c8f2e4 admin admin] PATH_INFO: `/v3/projects/bedc15e48c6e4781bcc72e1792862c9b/users/8225c49f1ab8403ca352084c3d715807/roles/8e2919e79df040e8926f21607818b58c` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:33:11.095968 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6506e64-4bdc-4f8a-afd2-c8dc35c8f2e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bedc15e48c6e4781bcc72e1792862c9b, user_id=8225c49f1ab8403ca352084c3d715807, role_id=8e2919e79df040e8926f21607818b58c)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:33:11.099176 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6506e64-4bdc-4f8a-afd2-c8dc35c8f2e4 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:33:11.100926 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:33:11.229980 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1392/2806] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Sep 23 15:33:11 2022] PUT /identity/v3/projects/bedc15e48c6e4781bcc72e1792862c9b/users/8225c49f1ab8403ca352084c3d715807/roles/8e2919e79df040e8926f21607818b58c => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:33:11.234089 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa78ad32-87bf-45c8-a0c3-ec62007f03d9 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:33:11.245496 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa78ad32-87bf-45c8-a0c3-ec62007f03d9 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:33:11.263232 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:33:11.305443 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36fdddb-96c2-4b62-b353-3936753181ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:33:11.305971 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36fdddb-96c2-4b62-b353-3936753181ad admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:33:11.306256 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36fdddb-96c2-4b62-b353-3936753181ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:33:11.306582 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36fdddb-96c2-4b62-b353-3936753181ad admin admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:33:11.307270 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36fdddb-96c2-4b62-b353-3936753181ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:33:11.309640 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36fdddb-96c2-4b62-b353-3936753181ad admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:33:11.321872 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1415/2807] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Sep 23 15:33:11 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:33:11.326097 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6506e64-4bdc-4f8a-afd2-c8dc35c8f2e4 admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:33:11.337344 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6506e64-4bdc-4f8a-afd2-c8dc35c8f2e4 admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:33:11.341961 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59204d4-ce3b-40b0-8c33-019cf9adb365 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:33:11.342610 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59204d4-ce3b-40b0-8c33-019cf9adb365 admin admin] REQUEST_METHOD: `PUT` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:33:11.342885 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59204d4-ce3b-40b0-8c33-019cf9adb365 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:33:11.343154 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59204d4-ce3b-40b0-8c33-019cf9adb365 admin admin] PATH_INFO: `/v3/projects/bedc15e48c6e4781bcc72e1792862c9b/users/8225c49f1ab8403ca352084c3d715807/roles/8e2919e79df040e8926f21607818b58c` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:33:11.345062 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59204d4-ce3b-40b0-8c33-019cf9adb365 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bedc15e48c6e4781bcc72e1792862c9b, user_id=8225c49f1ab8403ca352084c3d715807, role_id=8e2919e79df040e8926f21607818b58c)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:33:11.347663 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59204d4-ce3b-40b0-8c33-019cf9adb365 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:33:11.358074 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1393/2808] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Sep 23 15:33:11 2022] PUT /identity/v3/projects/bedc15e48c6e4781bcc72e1792862c9b/users/8225c49f1ab8403ca352084c3d715807/roles/8e2919e79df040e8926f21607818b58c => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:33:11.369625 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0c919e-33c8-4f75-b05a-c0c9eaae5022 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:33:11.369916 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0c919e-33c8-4f75-b05a-c0c9eaae5022 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:33:11.370198 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0c919e-33c8-4f75-b05a-c0c9eaae5022 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:33:11.399549 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-3c0c919e-33c8-4f75-b05a-c0c9eaae5022 None None] MFA Rules not processed for user `8225c49f1ab8403ca352084c3d715807`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:33:11.400932 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-3c0c919e-33c8-4f75-b05a-c0c9eaae5022 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:33:11.471492 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1416/2809] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:33:11 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:33:11.483290 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59204d4-ce3b-40b0-8c33-019cf9adb365 admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:33:11.485079 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-d59204d4-ce3b-40b0-8c33-019cf9adb365 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:33:11.548518 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59204d4-ce3b-40b0-8c33-019cf9adb365 admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:33:11.553174 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ed7c3d-b53c-4e8c-87f4-46f4815dff84 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feca6278be904ef8b0dc580a67e477f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:33:11.553783 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ed7c3d-b53c-4e8c-87f4-46f4815dff84 service cinder] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:33:11.554118 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ed7c3d-b53c-4e8c-87f4-46f4815dff84 service cinder] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:33:11.554388 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ed7c3d-b53c-4e8c-87f4-46f4815dff84 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:33:11.564773 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ed7c3d-b53c-4e8c-87f4-46f4815dff84 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:33:11.566131 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ed7c3d-b53c-4e8c-87f4-46f4815dff84 service cinder] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:33:11.576731 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1394/2810] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:33:11 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 23 15:33:11.735403 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0c919e-33c8-4f75-b05a-c0c9eaae5022 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:33:11.746682 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0c919e-33c8-4f75-b05a-c0c9eaae5022 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:33:11.750348 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d0dc4a-233b-41c5-b380-d3bb16579f32 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feca6278be904ef8b0dc580a67e477f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:33:11.750960 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d0dc4a-233b-41c5-b380-d3bb16579f32 service cinder] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:33:11.751243 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d0dc4a-233b-41c5-b380-d3bb16579f32 service cinder] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:33:11.751519 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d0dc4a-233b-41c5-b380-d3bb16579f32 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:33:11.810626 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d0dc4a-233b-41c5-b380-d3bb16579f32 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:33:11.812096 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d0dc4a-233b-41c5-b380-d3bb16579f32 service cinder] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:33:11.821322 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1417/2811] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:33:11 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:33:12.291121 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ed7c3d-b53c-4e8c-87f4-46f4815dff84 service cinder] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:33:12.355269 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ed7c3d-b53c-4e8c-87f4-46f4815dff84 service cinder] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:33:12.359802 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e024c5b-db0e-471a-99fb-07be6ebccf05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:33:12.360473 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e024c5b-db0e-471a-99fb-07be6ebccf05 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:33:12.360754 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e024c5b-db0e-471a-99fb-07be6ebccf05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:33:12.361051 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e024c5b-db0e-471a-99fb-07be6ebccf05 admin admin] PATH_INFO: `/v3/users/1279a2e4dd3d4de1af379a2fd6a6078b` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:33:12.362147 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e024c5b-db0e-471a-99fb-07be6ebccf05 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1279a2e4dd3d4de1af379a2fd6a6078b)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:33:12.365005 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e024c5b-db0e-471a-99fb-07be6ebccf05 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:33:12.546916 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-7e024c5b-db0e-471a-99fb-07be6ebccf05 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1279a2e4dd3d4de1af379a2fd6a6078b'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:33:12.561649 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-7e024c5b-db0e-471a-99fb-07be6ebccf05 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1279a2e4dd3d4de1af379a2fd6a6078b'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:33:12.570025 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-7e024c5b-db0e-471a-99fb-07be6ebccf05 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1279a2e4dd3d4de1af379a2fd6a6078b'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:33:12.646769 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-7e024c5b-db0e-471a-99fb-07be6ebccf05 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1279a2e4dd3d4de1af379a2fd6a6078b'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:33:12.647464 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-7e024c5b-db0e-471a-99fb-07be6ebccf05 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1279a2e4dd3d4de1af379a2fd6a6078b'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:33:12.654307 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1395/2812] 10.222.0.55 () {58 vars in 1224 bytes} [Fri Sep 23 15:33:12 2022] DELETE /identity/v3/users/1279a2e4dd3d4de1af379a2fd6a6078b => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:33:12.659297 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53659228-0da2-4e0b-96a1-0da906861e2d None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:33:12.659797 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53659228-0da2-4e0b-96a1-0da906861e2d None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:33:12.659900 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53659228-0da2-4e0b-96a1-0da906861e2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:33:12.680382 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-53659228-0da2-4e0b-96a1-0da906861e2d None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:33:12.681617 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-53659228-0da2-4e0b-96a1-0da906861e2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:33:12.717105 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:33:12.742953 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1418/2813] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:33:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:33:12.752976 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e024c5b-db0e-471a-99fb-07be6ebccf05 admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:33:12.754420 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-7e024c5b-db0e-471a-99fb-07be6ebccf05 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:33:12.760744 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:INSERT=3,keystone:SELECT=87,keystone:DELETE=10 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:33:12.816497 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e024c5b-db0e-471a-99fb-07be6ebccf05 admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:33:12.821133 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9976170a-635d-4b54-8652-29c9f27020ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd259178570864ffeb936ee10dfec6b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:33:12.821753 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9976170a-635d-4b54-8652-29c9f27020ca service neutron] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:33:12.822032 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9976170a-635d-4b54-8652-29c9f27020ca service neutron] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:33:12.822303 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9976170a-635d-4b54-8652-29c9f27020ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:33:12.832089 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9976170a-635d-4b54-8652-29c9f27020ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:33:12.834840 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9976170a-635d-4b54-8652-29c9f27020ca service neutron] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:33:12.845143 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1396/2814] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:33:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:33:13.276039 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53659228-0da2-4e0b-96a1-0da906861e2d None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:33:13.277575 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-53659228-0da2-4e0b-96a1-0da906861e2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:33:13.290710 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53659228-0da2-4e0b-96a1-0da906861e2d None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:33:13.295501 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73aa4978-9cb2-44ce-b961-53c621339856 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:33:13.296187 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73aa4978-9cb2-44ce-b961-53c621339856 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:33:13.296473 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73aa4978-9cb2-44ce-b961-53c621339856 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:33:13.296754 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73aa4978-9cb2-44ce-b961-53c621339856 admin admin] PATH_INFO: `/v3/projects/58ccddd1f0c549a2acc867aad5423f9f` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:33:13.297893 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73aa4978-9cb2-44ce-b961-53c621339856 admin admin] RBAC: Authorizing `identity:delete_project(project_id=58ccddd1f0c549a2acc867aad5423f9f)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:33:13.299814 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73aa4978-9cb2-44ce-b961-53c621339856 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:33:13.409742 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-73aa4978-9cb2-44ce-b961-53c621339856 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '58ccddd1f0c549a2acc867aad5423f9f', 'request_id': 'req-73aa4978-9cb2-44ce-b961-53c621339856'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:33:13.417018 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-73aa4978-9cb2-44ce-b961-53c621339856 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '58ccddd1f0c549a2acc867aad5423f9f', 'request_id': 'req-73aa4978-9cb2-44ce-b961-53c621339856'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:33:13.422844 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-73aa4978-9cb2-44ce-b961-53c621339856 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '58ccddd1f0c549a2acc867aad5423f9f', 'request_id': 'req-73aa4978-9cb2-44ce-b961-53c621339856'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:33:13.485475 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-73aa4978-9cb2-44ce-b961-53c621339856 admin admin] The token cache is being invalidate because project 58ccddd1f0c549a2acc867aad5423f9f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81994) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 15:33:13.485766 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-73aa4978-9cb2-44ce-b961-53c621339856 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:33:13.486406 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-73aa4978-9cb2-44ce-b961-53c621339856 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:33:13.488269 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1419/2815] 10.222.0.55 () {58 vars in 1233 bytes} [Fri Sep 23 15:33:13 2022] DELETE /identity/v3/projects/58ccddd1f0c549a2acc867aad5423f9f => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:33:13.492990 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9976170a-635d-4b54-8652-29c9f27020ca service neutron] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:33:13.503099 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9976170a-635d-4b54-8652-29c9f27020ca service neutron] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:33:13.565826 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23b86ec-e5db-48e5-bd93-9baece77ea31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:33:13.566373 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23b86ec-e5db-48e5-bd93-9baece77ea31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:33:13.566653 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23b86ec-e5db-48e5-bd93-9baece77ea31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:33:13.566928 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23b86ec-e5db-48e5-bd93-9baece77ea31 admin admin] PATH_INFO: `/v3/users/8225c49f1ab8403ca352084c3d715807` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:33:13.568206 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23b86ec-e5db-48e5-bd93-9baece77ea31 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8225c49f1ab8403ca352084c3d715807)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:33:13.570119 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23b86ec-e5db-48e5-bd93-9baece77ea31 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:33:13.719652 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-c23b86ec-e5db-48e5-bd93-9baece77ea31 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8225c49f1ab8403ca352084c3d715807'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:33:13.733388 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-c23b86ec-e5db-48e5-bd93-9baece77ea31 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8225c49f1ab8403ca352084c3d715807'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:33:13.746718 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-c23b86ec-e5db-48e5-bd93-9baece77ea31 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8225c49f1ab8403ca352084c3d715807'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:33:13.839264 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-c23b86ec-e5db-48e5-bd93-9baece77ea31 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8225c49f1ab8403ca352084c3d715807'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:33:13.839820 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-c23b86ec-e5db-48e5-bd93-9baece77ea31 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8225c49f1ab8403ca352084c3d715807'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:33:13.846244 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1397/2816] 10.222.0.55 () {58 vars in 1224 bytes} [Fri Sep 23 15:33:13 2022] DELETE /identity/v3/users/8225c49f1ab8403ca352084c3d715807 => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:33:14.264753 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73aa4978-9cb2-44ce-b961-53c621339856 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:33:14.269869 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-73aa4978-9cb2-44ce-b961-53c621339856 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:33:14.281297 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73aa4978-9cb2-44ce-b961-53c621339856 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:33:14.343104 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d1f576-0e15-4d46-a06c-757eb355abf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:33:14.343749 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d1f576-0e15-4d46-a06c-757eb355abf7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:33:14.344079 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d1f576-0e15-4d46-a06c-757eb355abf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:33:14.344393 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d1f576-0e15-4d46-a06c-757eb355abf7 admin admin] PATH_INFO: `/v3/projects/bedc15e48c6e4781bcc72e1792862c9b` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:33:14.345556 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d1f576-0e15-4d46-a06c-757eb355abf7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bedc15e48c6e4781bcc72e1792862c9b)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:33:14.347461 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d1f576-0e15-4d46-a06c-757eb355abf7 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:33:14.463831 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-28d1f576-0e15-4d46-a06c-757eb355abf7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bedc15e48c6e4781bcc72e1792862c9b', 'request_id': 'req-28d1f576-0e15-4d46-a06c-757eb355abf7'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:33:14.470432 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-28d1f576-0e15-4d46-a06c-757eb355abf7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bedc15e48c6e4781bcc72e1792862c9b', 'request_id': 'req-28d1f576-0e15-4d46-a06c-757eb355abf7'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:33:14.477243 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-28d1f576-0e15-4d46-a06c-757eb355abf7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bedc15e48c6e4781bcc72e1792862c9b', 'request_id': 'req-28d1f576-0e15-4d46-a06c-757eb355abf7'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:33:14.479877 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:33:14.587193 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-28d1f576-0e15-4d46-a06c-757eb355abf7 admin admin] The token cache is being invalidate because project bedc15e48c6e4781bcc72e1792862c9b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81994) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 15:33:14.587488 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-28d1f576-0e15-4d46-a06c-757eb355abf7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:33:14.588173 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-28d1f576-0e15-4d46-a06c-757eb355abf7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:33:14.589693 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1420/2817] 10.222.0.55 () {58 vars in 1233 bytes} [Fri Sep 23 15:33:14 2022] DELETE /identity/v3/projects/bedc15e48c6e4781bcc72e1792862c9b => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:33:14.594614 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94fb19c-201e-49fb-a95b-c49a3976d9f8 None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:33:14.599010 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94fb19c-201e-49fb-a95b-c49a3976d9f8 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:33:14.599010 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94fb19c-201e-49fb-a95b-c49a3976d9f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:33:14.617113 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-d94fb19c-201e-49fb-a95b-c49a3976d9f8 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:33:14.618341 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-d94fb19c-201e-49fb-a95b-c49a3976d9f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:33:14.639442 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:33:14.684120 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1398/2818] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:33:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:33:14.692138 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8aab3f1-fa03-42fe-9bb2-895f171b0ff0 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:33:14.694033 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8aab3f1-fa03-42fe-9bb2-895f171b0ff0 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:33:14.694322 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8aab3f1-fa03-42fe-9bb2-895f171b0ff0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:33:14.718969 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-a8aab3f1-fa03-42fe-9bb2-895f171b0ff0 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:33:14.720177 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-a8aab3f1-fa03-42fe-9bb2-895f171b0ff0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:33:14.729557 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1421/2819] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:33:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:33:14.734304 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94fb19c-201e-49fb-a95b-c49a3976d9f8 None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:33:14.747007 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94fb19c-201e-49fb-a95b-c49a3976d9f8 None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:33:14.751746 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba6415e-a311-4fe8-a11e-7a2a0b484e11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:33:14.752405 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba6415e-a311-4fe8-a11e-7a2a0b484e11 admin admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:33:14.752682 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba6415e-a311-4fe8-a11e-7a2a0b484e11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:33:14.752954 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba6415e-a311-4fe8-a11e-7a2a0b484e11 admin admin] PATH_INFO: `/v3/domains` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:33:14.753610 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba6415e-a311-4fe8-a11e-7a2a0b484e11 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:33:14.753917 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba6415e-a311-4fe8-a11e-7a2a0b484e11 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:33:14.756813 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba6415e-a311-4fe8-a11e-7a2a0b484e11 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:33:14.770004 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1399/2820] 10.222.0.55 () {58 vars in 1153 bytes} [Fri Sep 23 15:33:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:33:14.774304 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8aab3f1-fa03-42fe-9bb2-895f171b0ff0 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:33:14.784337 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8aab3f1-fa03-42fe-9bb2-895f171b0ff0 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:33:14.790069 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e111020b-c52d-440b-8a99-3bbe315c4884 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:33:14.790698 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e111020b-c52d-440b-8a99-3bbe315c4884 admin admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:33:14.790980 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e111020b-c52d-440b-8a99-3bbe315c4884 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:33:14.791254 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e111020b-c52d-440b-8a99-3bbe315c4884 admin admin] PATH_INFO: `/v3/projects` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:33:14.791900 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e111020b-c52d-440b-8a99-3bbe315c4884 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:33:14.793931 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e111020b-c52d-440b-8a99-3bbe315c4884 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:33:14.904606 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1422/2821] 10.222.0.55 () {60 vars in 1153 bytes} [Fri Sep 23 15:33:14 2022] POST /identity/v3/projects => generated 383 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 23 15:33:14.908862 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba6415e-a311-4fe8-a11e-7a2a0b484e11 admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:33:14.919798 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba6415e-a311-4fe8-a11e-7a2a0b484e11 admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:33:14.978433 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d04842-7c4d-46ce-83fc-3728b0ff2a74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:33:14.979056 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d04842-7c4d-46ce-83fc-3728b0ff2a74 admin admin] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:33:14.979337 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d04842-7c4d-46ce-83fc-3728b0ff2a74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:33:14.979606 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d04842-7c4d-46ce-83fc-3728b0ff2a74 admin admin] PATH_INFO: `/v3/users` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:33:14.980273 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d04842-7c4d-46ce-83fc-3728b0ff2a74 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:33:14.982189 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d04842-7c4d-46ce-83fc-3728b0ff2a74 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:33:15.159972 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1400/2822] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Sep 23 15:33:14 2022] POST /identity/v3/users => generated 292 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:33:15.164772 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e111020b-c52d-440b-8a99-3bbe315c4884 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:33:15.175427 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e111020b-c52d-440b-8a99-3bbe315c4884 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:33:15.181126 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3efafcc-1f5a-4306-8fec-ca156d64a9e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:33:15.181739 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3efafcc-1f5a-4306-8fec-ca156d64a9e6 admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:33:15.182017 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3efafcc-1f5a-4306-8fec-ca156d64a9e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:33:15.182287 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3efafcc-1f5a-4306-8fec-ca156d64a9e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:33:15.182948 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3efafcc-1f5a-4306-8fec-ca156d64a9e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:33:15.184568 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3efafcc-1f5a-4306-8fec-ca156d64a9e6 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:33:15.195584 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1423/2823] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Sep 23 15:33:15 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:33:15.199714 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d04842-7c4d-46ce-83fc-3728b0ff2a74 admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:33:15.211504 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d04842-7c4d-46ce-83fc-3728b0ff2a74 admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:33:15.217028 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae10af5-8893-49c0-94ba-215de32560d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:33:15.217827 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae10af5-8893-49c0-94ba-215de32560d1 admin admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:33:15.218318 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae10af5-8893-49c0-94ba-215de32560d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:33:15.218785 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae10af5-8893-49c0-94ba-215de32560d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:33:15.219760 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae10af5-8893-49c0-94ba-215de32560d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:33:15.221381 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae10af5-8893-49c0-94ba-215de32560d1 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:33:15.233856 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1401/2824] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Sep 23 15:33:15 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:33:15.242249 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3efafcc-1f5a-4306-8fec-ca156d64a9e6 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:33:15.257457 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3efafcc-1f5a-4306-8fec-ca156d64a9e6 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:33:15.264400 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ff983d-58b1-45f8-9959-62460e5abc24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:33:15.265073 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ff983d-58b1-45f8-9959-62460e5abc24 admin admin] REQUEST_METHOD: `PUT` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:33:15.265826 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ff983d-58b1-45f8-9959-62460e5abc24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:33:15.265826 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ff983d-58b1-45f8-9959-62460e5abc24 admin admin] PATH_INFO: `/v3/projects/2af71b98b0f54b6bb49092ffa4591596/users/d149d404965a4228aed2d51f1fe58893/roles/af257be2dee54d6b942f58650196582c` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:33:15.283902 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ff983d-58b1-45f8-9959-62460e5abc24 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2af71b98b0f54b6bb49092ffa4591596, user_id=d149d404965a4228aed2d51f1fe58893, role_id=af257be2dee54d6b942f58650196582c)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:33:15.287324 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ff983d-58b1-45f8-9959-62460e5abc24 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:33:15.377393 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1424/2825] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Sep 23 15:33:15 2022] PUT /identity/v3/projects/2af71b98b0f54b6bb49092ffa4591596/users/d149d404965a4228aed2d51f1fe58893/roles/af257be2dee54d6b942f58650196582c => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:33:15.389683 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831e724b-25a5-4d6f-8f36-854abb6394b2 None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:33:15.390560 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831e724b-25a5-4d6f-8f36-854abb6394b2 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:33:15.390840 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831e724b-25a5-4d6f-8f36-854abb6394b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:33:15.425115 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-831e724b-25a5-4d6f-8f36-854abb6394b2 None None] MFA Rules not processed for user `d149d404965a4228aed2d51f1fe58893`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:33:15.426349 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-831e724b-25a5-4d6f-8f36-854abb6394b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:33:15.495325 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1402/2826] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:33:15 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:33:15.500187 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ff983d-58b1-45f8-9959-62460e5abc24 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:33:15.566800 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ff983d-58b1-45f8-9959-62460e5abc24 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:33:15.580104 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d65d517-01fb-4f79-a5fe-8a7c29f900df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:33:15.583795 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d65d517-01fb-4f79-a5fe-8a7c29f900df admin admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:33:15.583795 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d65d517-01fb-4f79-a5fe-8a7c29f900df admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:33:15.583795 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d65d517-01fb-4f79-a5fe-8a7c29f900df admin admin] PATH_INFO: `/v3/projects` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:33:15.583795 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d65d517-01fb-4f79-a5fe-8a7c29f900df admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:33:15.584650 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d65d517-01fb-4f79-a5fe-8a7c29f900df admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:33:15.671179 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1425/2827] 10.222.0.55 () {60 vars in 1153 bytes} [Fri Sep 23 15:33:15 2022] POST /identity/v3/projects => generated 381 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:33:15.674408 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831e724b-25a5-4d6f-8f36-854abb6394b2 None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:33:15.685993 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831e724b-25a5-4d6f-8f36-854abb6394b2 None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:33:15.703566 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:33:15.746974 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0d9c42-7bd8-4f7f-b579-e78801032c6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:33:15.747505 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0d9c42-7bd8-4f7f-b579-e78801032c6e admin admin] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:33:15.747783 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0d9c42-7bd8-4f7f-b579-e78801032c6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:33:15.748087 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0d9c42-7bd8-4f7f-b579-e78801032c6e admin admin] PATH_INFO: `/v3/users` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:33:15.748715 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0d9c42-7bd8-4f7f-b579-e78801032c6e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:33:15.750633 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0d9c42-7bd8-4f7f-b579-e78801032c6e admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:33:15.823355 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1403/2828] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Sep 23 15:33:15 2022] POST /identity/v3/users => generated 297 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:33:15.827910 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d65d517-01fb-4f79-a5fe-8a7c29f900df admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:33:15.838749 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d65d517-01fb-4f79-a5fe-8a7c29f900df admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:33:15.843402 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f419f418-e438-4762-b318-c4c3caa8ec68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:33:15.844096 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f419f418-e438-4762-b318-c4c3caa8ec68 admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:33:15.844382 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f419f418-e438-4762-b318-c4c3caa8ec68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:33:15.844656 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f419f418-e438-4762-b318-c4c3caa8ec68 admin admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:33:15.845293 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f419f418-e438-4762-b318-c4c3caa8ec68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:33:15.846896 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f419f418-e438-4762-b318-c4c3caa8ec68 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:33:15.858751 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1426/2829] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Sep 23 15:33:15 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 23 15:33:15.863441 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0d9c42-7bd8-4f7f-b579-e78801032c6e admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:33:15.874203 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0d9c42-7bd8-4f7f-b579-e78801032c6e admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:33:15.878874 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2504ff-caed-40df-8ab7-f8065a5a80b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:33:15.879643 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2504ff-caed-40df-8ab7-f8065a5a80b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:33:15.879931 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2504ff-caed-40df-8ab7-f8065a5a80b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:33:15.880351 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2504ff-caed-40df-8ab7-f8065a5a80b2 admin admin] PATH_INFO: `/v3/projects/72e88824922541d99889cff2bb6c329f/users/f79236cbb8a14008bc0de7e31b2fd95a/roles/8e2919e79df040e8926f21607818b58c` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:33:15.895385 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2504ff-caed-40df-8ab7-f8065a5a80b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=72e88824922541d99889cff2bb6c329f, user_id=f79236cbb8a14008bc0de7e31b2fd95a, role_id=8e2919e79df040e8926f21607818b58c)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:33:15.898367 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2504ff-caed-40df-8ab7-f8065a5a80b2 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:33:15.961145 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1404/2830] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Sep 23 15:33:15 2022] PUT /identity/v3/projects/72e88824922541d99889cff2bb6c329f/users/f79236cbb8a14008bc0de7e31b2fd95a/roles/8e2919e79df040e8926f21607818b58c => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:33:15.965362 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f419f418-e438-4762-b318-c4c3caa8ec68 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:33:15.976627 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f419f418-e438-4762-b318-c4c3caa8ec68 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:33:16.035229 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e94ab4b-9137-49b3-9d76-904c33bac774 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:33:16.035752 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e94ab4b-9137-49b3-9d76-904c33bac774 admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:33:16.036098 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e94ab4b-9137-49b3-9d76-904c33bac774 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:33:16.037024 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e94ab4b-9137-49b3-9d76-904c33bac774 admin admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:33:16.037660 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e94ab4b-9137-49b3-9d76-904c33bac774 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:33:16.039253 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e94ab4b-9137-49b3-9d76-904c33bac774 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:33:16.044443 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:33:16.049419 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1427/2831] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Sep 23 15:33:15 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 23 15:33:16.054124 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2504ff-caed-40df-8ab7-f8065a5a80b2 admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:33:16.065066 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2504ff-caed-40df-8ab7-f8065a5a80b2 admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:33:16.069791 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc18e2f-7513-475d-ace4-4a8673d4c072 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:33:16.070445 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc18e2f-7513-475d-ace4-4a8673d4c072 admin admin] REQUEST_METHOD: `PUT` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:33:16.070723 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc18e2f-7513-475d-ace4-4a8673d4c072 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:33:16.070997 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc18e2f-7513-475d-ace4-4a8673d4c072 admin admin] PATH_INFO: `/v3/projects/72e88824922541d99889cff2bb6c329f/users/f79236cbb8a14008bc0de7e31b2fd95a/roles/8e2919e79df040e8926f21607818b58c` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:33:16.072918 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc18e2f-7513-475d-ace4-4a8673d4c072 admin admin] RBAC: Authorizing `identity:create_grant(project_id=72e88824922541d99889cff2bb6c329f, user_id=f79236cbb8a14008bc0de7e31b2fd95a, role_id=8e2919e79df040e8926f21607818b58c)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:33:16.075524 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc18e2f-7513-475d-ace4-4a8673d4c072 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:33:16.087055 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1405/2832] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Sep 23 15:33:16 2022] PUT /identity/v3/projects/72e88824922541d99889cff2bb6c329f/users/f79236cbb8a14008bc0de7e31b2fd95a/roles/8e2919e79df040e8926f21607818b58c => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:33:16.097516 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e6fb7b-70d0-458e-9d9d-0288a26c2ea5 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:33:16.097807 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e6fb7b-70d0-458e-9d9d-0288a26c2ea5 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:33:16.098087 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e6fb7b-70d0-458e-9d9d-0288a26c2ea5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:33:16.126886 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-e9e6fb7b-70d0-458e-9d9d-0288a26c2ea5 None None] MFA Rules not processed for user `f79236cbb8a14008bc0de7e31b2fd95a`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:33:16.128148 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-e9e6fb7b-70d0-458e-9d9d-0288a26c2ea5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:33:16.197961 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1428/2833] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:33:16 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:33:16.209311 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc18e2f-7513-475d-ace4-4a8673d4c072 admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:33:16.211014 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-3dc18e2f-7513-475d-ace4-4a8673d4c072 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:33:16.268447 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc18e2f-7513-475d-ace4-4a8673d4c072 admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:33:16.272700 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9eb5d0-3908-42da-b1e7-686385baa282 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feca6278be904ef8b0dc580a67e477f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:33:16.273307 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9eb5d0-3908-42da-b1e7-686385baa282 service cinder] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:33:16.273590 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9eb5d0-3908-42da-b1e7-686385baa282 service cinder] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:33:16.273896 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9eb5d0-3908-42da-b1e7-686385baa282 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:33:16.330718 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9eb5d0-3908-42da-b1e7-686385baa282 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:33:16.332092 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9eb5d0-3908-42da-b1e7-686385baa282 service cinder] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:33:16.341556 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1406/2834] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:33:16 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:33:21.789058 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e6fb7b-70d0-458e-9d9d-0288a26c2ea5 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:33:21.790532 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-e9e6fb7b-70d0-458e-9d9d-0288a26c2ea5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:33:21.803377 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e6fb7b-70d0-458e-9d9d-0288a26c2ea5 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:33:21.806997 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0725ce1-71a5-4379-9d29-96cfbe5808a1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feca6278be904ef8b0dc580a67e477f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:33:21.807641 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0725ce1-71a5-4379-9d29-96cfbe5808a1 service cinder] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:33:21.807932 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0725ce1-71a5-4379-9d29-96cfbe5808a1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:33:21.808253 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0725ce1-71a5-4379-9d29-96cfbe5808a1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:33:21.817958 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0725ce1-71a5-4379-9d29-96cfbe5808a1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:33:21.820062 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0725ce1-71a5-4379-9d29-96cfbe5808a1 service cinder] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:33:21.829871 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1429/2835] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:33:21 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:33:26.337719 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=83,keystone:INSERT=5 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:33:27.648795 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9eb5d0-3908-42da-b1e7-686385baa282 service cinder] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:33:27.711740 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9eb5d0-3908-42da-b1e7-686385baa282 service cinder] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:33:27.716347 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018bf13b-e707-4d5a-9a46-38864f6376f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:33:27.716981 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018bf13b-e707-4d5a-9a46-38864f6376f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:33:27.717263 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018bf13b-e707-4d5a-9a46-38864f6376f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:33:27.717537 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018bf13b-e707-4d5a-9a46-38864f6376f0 admin admin] PATH_INFO: `/v3/users/d149d404965a4228aed2d51f1fe58893` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:33:27.718651 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018bf13b-e707-4d5a-9a46-38864f6376f0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d149d404965a4228aed2d51f1fe58893)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:33:27.720609 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018bf13b-e707-4d5a-9a46-38864f6376f0 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:33:27.917347 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-018bf13b-e707-4d5a-9a46-38864f6376f0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd149d404965a4228aed2d51f1fe58893'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:33:27.930620 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-018bf13b-e707-4d5a-9a46-38864f6376f0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd149d404965a4228aed2d51f1fe58893'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:33:27.937896 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-018bf13b-e707-4d5a-9a46-38864f6376f0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd149d404965a4228aed2d51f1fe58893'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:33:28.022722 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-018bf13b-e707-4d5a-9a46-38864f6376f0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd149d404965a4228aed2d51f1fe58893'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:33:28.023422 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-018bf13b-e707-4d5a-9a46-38864f6376f0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd149d404965a4228aed2d51f1fe58893'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:33:28.030388 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1407/2836] 10.222.0.55 () {58 vars in 1224 bytes} [Fri Sep 23 15:33:27 2022] DELETE /identity/v3/users/d149d404965a4228aed2d51f1fe58893 => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:33:28.034907 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754f384f-09fc-4f1f-9c62-b4bbf05631ea None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:33:28.035394 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754f384f-09fc-4f1f-9c62-b4bbf05631ea None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:33:28.035522 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754f384f-09fc-4f1f-9c62-b4bbf05631ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:33:28.055833 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-754f384f-09fc-4f1f-9c62-b4bbf05631ea None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:33:28.057076 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-754f384f-09fc-4f1f-9c62-b4bbf05631ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:33:28.122476 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1430/2837] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:33:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:33:28.133144 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018bf13b-e707-4d5a-9a46-38864f6376f0 admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:33:28.134569 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-018bf13b-e707-4d5a-9a46-38864f6376f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:33:28.194462 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018bf13b-e707-4d5a-9a46-38864f6376f0 admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:33:28.198104 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd83469-dea1-4a65-a73f-1ff70213e7ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd259178570864ffeb936ee10dfec6b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:33:28.198716 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd83469-dea1-4a65-a73f-1ff70213e7ae service neutron] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:33:28.198995 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd83469-dea1-4a65-a73f-1ff70213e7ae service neutron] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:33:28.199269 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd83469-dea1-4a65-a73f-1ff70213e7ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:33:28.209682 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd83469-dea1-4a65-a73f-1ff70213e7ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:33:28.211631 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd83469-dea1-4a65-a73f-1ff70213e7ae service neutron] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:33:28.221075 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1408/2838] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:33:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:33:28.669732 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754f384f-09fc-4f1f-9c62-b4bbf05631ea None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:33:28.671175 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-754f384f-09fc-4f1f-9c62-b4bbf05631ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:33:28.683862 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754f384f-09fc-4f1f-9c62-b4bbf05631ea None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:33:28.689098 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13e4ea4-19c8-4fce-96b8-ad74b03a89c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:33:28.693081 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13e4ea4-19c8-4fce-96b8-ad74b03a89c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:33:28.693081 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13e4ea4-19c8-4fce-96b8-ad74b03a89c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:33:28.693081 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13e4ea4-19c8-4fce-96b8-ad74b03a89c3 admin admin] PATH_INFO: `/v3/projects/2af71b98b0f54b6bb49092ffa4591596` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:33:28.693081 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13e4ea4-19c8-4fce-96b8-ad74b03a89c3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2af71b98b0f54b6bb49092ffa4591596)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:33:28.693883 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13e4ea4-19c8-4fce-96b8-ad74b03a89c3 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:33:28.824323 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-c13e4ea4-19c8-4fce-96b8-ad74b03a89c3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2af71b98b0f54b6bb49092ffa4591596', 'request_id': 'req-c13e4ea4-19c8-4fce-96b8-ad74b03a89c3'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:33:28.830730 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-c13e4ea4-19c8-4fce-96b8-ad74b03a89c3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2af71b98b0f54b6bb49092ffa4591596', 'request_id': 'req-c13e4ea4-19c8-4fce-96b8-ad74b03a89c3'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:33:28.836871 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-c13e4ea4-19c8-4fce-96b8-ad74b03a89c3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2af71b98b0f54b6bb49092ffa4591596', 'request_id': 'req-c13e4ea4-19c8-4fce-96b8-ad74b03a89c3'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:33:28.840791 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:33:28.948250 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-c13e4ea4-19c8-4fce-96b8-ad74b03a89c3 admin admin] The token cache is being invalidate because project 2af71b98b0f54b6bb49092ffa4591596 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81994) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 15:33:28.948546 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-c13e4ea4-19c8-4fce-96b8-ad74b03a89c3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:33:28.949317 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-c13e4ea4-19c8-4fce-96b8-ad74b03a89c3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:33:28.950859 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1431/2839] 10.222.0.55 () {58 vars in 1233 bytes} [Fri Sep 23 15:33:28 2022] DELETE /identity/v3/projects/2af71b98b0f54b6bb49092ffa4591596 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:33:28.955006 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd83469-dea1-4a65-a73f-1ff70213e7ae service neutron] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:33:28.965850 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd83469-dea1-4a65-a73f-1ff70213e7ae service neutron] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:33:28.986148 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:33:29.028198 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183f3a07-9b03-4e9a-be1e-1d24852dadb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:33:29.028942 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183f3a07-9b03-4e9a-be1e-1d24852dadb1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:33:29.029221 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183f3a07-9b03-4e9a-be1e-1d24852dadb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:33:29.029493 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183f3a07-9b03-4e9a-be1e-1d24852dadb1 admin admin] PATH_INFO: `/v3/users/f79236cbb8a14008bc0de7e31b2fd95a` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:33:29.030697 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183f3a07-9b03-4e9a-be1e-1d24852dadb1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f79236cbb8a14008bc0de7e31b2fd95a)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:33:29.032745 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183f3a07-9b03-4e9a-be1e-1d24852dadb1 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:33:29.244109 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-183f3a07-9b03-4e9a-be1e-1d24852dadb1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f79236cbb8a14008bc0de7e31b2fd95a'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:33:29.257056 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-183f3a07-9b03-4e9a-be1e-1d24852dadb1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f79236cbb8a14008bc0de7e31b2fd95a'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:33:29.264321 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-183f3a07-9b03-4e9a-be1e-1d24852dadb1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f79236cbb8a14008bc0de7e31b2fd95a'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:33:29.313866 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-183f3a07-9b03-4e9a-be1e-1d24852dadb1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f79236cbb8a14008bc0de7e31b2fd95a'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:33:29.314499 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-183f3a07-9b03-4e9a-be1e-1d24852dadb1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f79236cbb8a14008bc0de7e31b2fd95a'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:33:29.324464 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1409/2840] 10.222.0.55 () {58 vars in 1224 bytes} [Fri Sep 23 15:33:28 2022] DELETE /identity/v3/users/f79236cbb8a14008bc0de7e31b2fd95a => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 15:33:29.749227 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13e4ea4-19c8-4fce-96b8-ad74b03a89c3 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:33:29.750700 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-c13e4ea4-19c8-4fce-96b8-ad74b03a89c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:33:29.766447 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13e4ea4-19c8-4fce-96b8-ad74b03a89c3 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:33:29.833588 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a102b54-78fe-4206-808f-05c89968c0c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:33:29.834235 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a102b54-78fe-4206-808f-05c89968c0c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:33:29.834518 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a102b54-78fe-4206-808f-05c89968c0c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:33:29.834791 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a102b54-78fe-4206-808f-05c89968c0c4 admin admin] PATH_INFO: `/v3/projects/72e88824922541d99889cff2bb6c329f` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:33:29.836059 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a102b54-78fe-4206-808f-05c89968c0c4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=72e88824922541d99889cff2bb6c329f)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:33:29.837972 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a102b54-78fe-4206-808f-05c89968c0c4 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:33:29.950511 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-3a102b54-78fe-4206-808f-05c89968c0c4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '72e88824922541d99889cff2bb6c329f', 'request_id': 'req-3a102b54-78fe-4206-808f-05c89968c0c4'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:33:29.957146 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-3a102b54-78fe-4206-808f-05c89968c0c4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '72e88824922541d99889cff2bb6c329f', 'request_id': 'req-3a102b54-78fe-4206-808f-05c89968c0c4'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:33:29.964217 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-3a102b54-78fe-4206-808f-05c89968c0c4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '72e88824922541d99889cff2bb6c329f', 'request_id': 'req-3a102b54-78fe-4206-808f-05c89968c0c4'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:33:30.022939 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-3a102b54-78fe-4206-808f-05c89968c0c4 admin admin] The token cache is being invalidate because project 72e88824922541d99889cff2bb6c329f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81994) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 15:33:30.023237 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-3a102b54-78fe-4206-808f-05c89968c0c4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:33:30.023878 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-3a102b54-78fe-4206-808f-05c89968c0c4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:33:30.025474 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1432/2841] 10.222.0.55 () {58 vars in 1233 bytes} [Fri Sep 23 15:33:29 2022] DELETE /identity/v3/projects/72e88824922541d99889cff2bb6c329f => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:33:30.034168 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9f857a-455e-46ef-873b-ae78a4f20cc5 None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:33:30.034168 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9f857a-455e-46ef-873b-ae78a4f20cc5 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:33:30.034168 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9f857a-455e-46ef-873b-ae78a4f20cc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:33:30.052868 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-1b9f857a-455e-46ef-873b-ae78a4f20cc5 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:33:30.054099 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-1b9f857a-455e-46ef-873b-ae78a4f20cc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:33:30.118459 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1410/2842] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:33:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:33:30.184087 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350ac8e9-a992-448e-a284-f6b598934e48 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:33:30.184378 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350ac8e9-a992-448e-a284-f6b598934e48 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:33:30.184656 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350ac8e9-a992-448e-a284-f6b598934e48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:33:30.209136 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-350ac8e9-a992-448e-a284-f6b598934e48 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:33:30.210475 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-350ac8e9-a992-448e-a284-f6b598934e48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:33:30.219907 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1433/2843] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:33:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:33:30.224357 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9f857a-455e-46ef-873b-ae78a4f20cc5 None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:33:30.234252 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9f857a-455e-46ef-873b-ae78a4f20cc5 None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:33:30.239809 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5fd711-af3f-48fe-adaa-953396328243 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:33:30.240647 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5fd711-af3f-48fe-adaa-953396328243 admin admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:33:30.240930 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5fd711-af3f-48fe-adaa-953396328243 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:33:30.241204 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5fd711-af3f-48fe-adaa-953396328243 admin admin] PATH_INFO: `/v3/domains` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:33:30.241857 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5fd711-af3f-48fe-adaa-953396328243 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:33:30.242398 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5fd711-af3f-48fe-adaa-953396328243 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:33:30.243772 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5fd711-af3f-48fe-adaa-953396328243 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:33:30.256821 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1411/2844] 10.222.0.55 () {58 vars in 1153 bytes} [Fri Sep 23 15:33:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:33:30.261149 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350ac8e9-a992-448e-a284-f6b598934e48 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:33:30.270201 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350ac8e9-a992-448e-a284-f6b598934e48 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:33:30.274944 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0390e57a-e358-4627-87c0-e0e0af7e2ca2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:33:30.275540 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0390e57a-e358-4627-87c0-e0e0af7e2ca2 admin admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:33:30.275818 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0390e57a-e358-4627-87c0-e0e0af7e2ca2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:33:30.276118 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0390e57a-e358-4627-87c0-e0e0af7e2ca2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:33:30.276750 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0390e57a-e358-4627-87c0-e0e0af7e2ca2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:33:30.278618 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0390e57a-e358-4627-87c0-e0e0af7e2ca2 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:33:30.349611 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1434/2845] 10.222.0.55 () {60 vars in 1153 bytes} [Fri Sep 23 15:33:30 2022] POST /identity/v3/projects => generated 375 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:33:30.354014 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5fd711-af3f-48fe-adaa-953396328243 admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:33:30.365866 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5fd711-af3f-48fe-adaa-953396328243 admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:33:30.392656 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:33:30.422416 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a09dc9-e99d-410e-b437-5e8ca8606666 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:33:30.423023 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a09dc9-e99d-410e-b437-5e8ca8606666 admin admin] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:33:30.423307 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a09dc9-e99d-410e-b437-5e8ca8606666 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:33:30.423590 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a09dc9-e99d-410e-b437-5e8ca8606666 admin admin] PATH_INFO: `/v3/users` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:33:30.424253 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a09dc9-e99d-410e-b437-5e8ca8606666 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:33:30.426202 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a09dc9-e99d-410e-b437-5e8ca8606666 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:33:30.537410 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1412/2846] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Sep 23 15:33:30 2022] POST /identity/v3/users => generated 288 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:33:30.542180 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0390e57a-e358-4627-87c0-e0e0af7e2ca2 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:33:30.553273 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0390e57a-e358-4627-87c0-e0e0af7e2ca2 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:33:30.557852 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9147aba0-c846-4131-b413-9af6c4064ad8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:33:30.558450 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9147aba0-c846-4131-b413-9af6c4064ad8 admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:33:30.558732 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9147aba0-c846-4131-b413-9af6c4064ad8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:33:30.559002 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9147aba0-c846-4131-b413-9af6c4064ad8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:33:30.559642 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9147aba0-c846-4131-b413-9af6c4064ad8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:33:30.561268 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9147aba0-c846-4131-b413-9af6c4064ad8 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:33:30.571934 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1435/2847] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Sep 23 15:33:30 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:33:30.576134 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a09dc9-e99d-410e-b437-5e8ca8606666 admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:33:30.586380 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a09dc9-e99d-410e-b437-5e8ca8606666 admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:33:30.591002 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a1c3f9-e8bd-443b-abf0-06eab8009e39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:33:30.591597 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a1c3f9-e8bd-443b-abf0-06eab8009e39 admin admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:33:30.591882 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a1c3f9-e8bd-443b-abf0-06eab8009e39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:33:30.592189 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a1c3f9-e8bd-443b-abf0-06eab8009e39 admin admin] PATH_INFO: `/v3/roles` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:33:30.592829 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a1c3f9-e8bd-443b-abf0-06eab8009e39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:33:30.594592 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a1c3f9-e8bd-443b-abf0-06eab8009e39 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:33:30.605848 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1413/2848] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Sep 23 15:33:30 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:33:30.610073 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9147aba0-c846-4131-b413-9af6c4064ad8 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:33:30.621380 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9147aba0-c846-4131-b413-9af6c4064ad8 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:33:30.626062 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db488f4-d0e9-4ea2-9c47-232af48968c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:33:30.626719 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db488f4-d0e9-4ea2-9c47-232af48968c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:33:30.627023 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db488f4-d0e9-4ea2-9c47-232af48968c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:33:30.627302 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db488f4-d0e9-4ea2-9c47-232af48968c3 admin admin] PATH_INFO: `/v3/projects/f6a0ed4c4c17430fab1dadfc5516fab9/users/1c2ebc6c8a124ac198ee792833a458e2/roles/af257be2dee54d6b942f58650196582c` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:33:30.642608 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db488f4-d0e9-4ea2-9c47-232af48968c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f6a0ed4c4c17430fab1dadfc5516fab9, user_id=1c2ebc6c8a124ac198ee792833a458e2, role_id=af257be2dee54d6b942f58650196582c)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:33:30.645652 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db488f4-d0e9-4ea2-9c47-232af48968c3 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:33:30.719903 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1436/2849] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Sep 23 15:33:30 2022] PUT /identity/v3/projects/f6a0ed4c4c17430fab1dadfc5516fab9/users/1c2ebc6c8a124ac198ee792833a458e2/roles/af257be2dee54d6b942f58650196582c => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:33:30.731991 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed94005e-c1f9-42b9-9619-2e5ea80197f8 None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:33:30.732342 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed94005e-c1f9-42b9-9619-2e5ea80197f8 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:33:30.732699 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed94005e-c1f9-42b9-9619-2e5ea80197f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:33:30.762647 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-ed94005e-c1f9-42b9-9619-2e5ea80197f8 None None] MFA Rules not processed for user `1c2ebc6c8a124ac198ee792833a458e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:33:30.763949 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-ed94005e-c1f9-42b9-9619-2e5ea80197f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:33:30.832690 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1414/2850] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:33:30 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:33:30.844450 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db488f4-d0e9-4ea2-9c47-232af48968c3 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:33:30.847532 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-8db488f4-d0e9-4ea2-9c47-232af48968c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:33:30.876472 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:33:30.907239 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db488f4-d0e9-4ea2-9c47-232af48968c3 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:33:30.911465 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3d008e-8b51-4f31-9b7f-aa9f78e8d5ae service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feca6278be904ef8b0dc580a67e477f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:33:30.912180 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3d008e-8b51-4f31-9b7f-aa9f78e8d5ae service cinder] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:33:30.912460 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3d008e-8b51-4f31-9b7f-aa9f78e8d5ae service cinder] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:33:30.912821 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3d008e-8b51-4f31-9b7f-aa9f78e8d5ae service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:33:30.921561 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3d008e-8b51-4f31-9b7f-aa9f78e8d5ae service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:33:30.922998 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3d008e-8b51-4f31-9b7f-aa9f78e8d5ae service cinder] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:33:30.933540 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1437/2851] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:33:30 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 23 15:33:35.162003 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed94005e-c1f9-42b9-9619-2e5ea80197f8 None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:33:35.227509 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed94005e-c1f9-42b9-9619-2e5ea80197f8 None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:33:35.233128 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fb45bd-4030-40fb-96fd-551c19ab4e85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:33:35.233771 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fb45bd-4030-40fb-96fd-551c19ab4e85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:33:35.234045 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fb45bd-4030-40fb-96fd-551c19ab4e85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:33:35.234314 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fb45bd-4030-40fb-96fd-551c19ab4e85 admin admin] PATH_INFO: `/v3/users/1c2ebc6c8a124ac198ee792833a458e2` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:33:35.235685 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fb45bd-4030-40fb-96fd-551c19ab4e85 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c2ebc6c8a124ac198ee792833a458e2)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:33:35.238693 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fb45bd-4030-40fb-96fd-551c19ab4e85 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:33:35.449099 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:33:35.455288 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-f2fb45bd-4030-40fb-96fd-551c19ab4e85 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c2ebc6c8a124ac198ee792833a458e2'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:33:35.469673 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-f2fb45bd-4030-40fb-96fd-551c19ab4e85 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c2ebc6c8a124ac198ee792833a458e2'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:33:35.477734 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-f2fb45bd-4030-40fb-96fd-551c19ab4e85 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c2ebc6c8a124ac198ee792833a458e2'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:33:35.554064 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-f2fb45bd-4030-40fb-96fd-551c19ab4e85 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c2ebc6c8a124ac198ee792833a458e2'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:33:35.554695 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-f2fb45bd-4030-40fb-96fd-551c19ab4e85 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c2ebc6c8a124ac198ee792833a458e2'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:33:35.562603 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1415/2852] 10.222.0.55 () {58 vars in 1224 bytes} [Fri Sep 23 15:33:35 2022] DELETE /identity/v3/users/1c2ebc6c8a124ac198ee792833a458e2 => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:33:35.567231 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d825000c-c6cd-4d7a-b047-0081fa107d30 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:33:35.568309 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d825000c-c6cd-4d7a-b047-0081fa107d30 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:33:35.568465 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d825000c-c6cd-4d7a-b047-0081fa107d30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:33:35.589438 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-d825000c-c6cd-4d7a-b047-0081fa107d30 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:33:35.590669 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-d825000c-c6cd-4d7a-b047-0081fa107d30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:33:35.657947 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1438/2853] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:33:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:33:35.671775 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fb45bd-4030-40fb-96fd-551c19ab4e85 admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:33:35.673829 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-f2fb45bd-4030-40fb-96fd-551c19ab4e85 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:33:35.742518 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fb45bd-4030-40fb-96fd-551c19ab4e85 admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:33:35.746168 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fed8a6-64ff-464f-8789-6a60ab37c8fa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd259178570864ffeb936ee10dfec6b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:33:35.746770 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fed8a6-64ff-464f-8789-6a60ab37c8fa service neutron] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:33:35.747042 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fed8a6-64ff-464f-8789-6a60ab37c8fa service neutron] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:33:35.747310 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fed8a6-64ff-464f-8789-6a60ab37c8fa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:33:35.757781 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fed8a6-64ff-464f-8789-6a60ab37c8fa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:33:35.759248 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fed8a6-64ff-464f-8789-6a60ab37c8fa service neutron] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:33:35.769358 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1416/2854] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:33:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:33:36.321370 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d825000c-c6cd-4d7a-b047-0081fa107d30 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:33:36.323047 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-d825000c-c6cd-4d7a-b047-0081fa107d30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:33:36.337636 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d825000c-c6cd-4d7a-b047-0081fa107d30 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:33:36.342631 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a18daf8-5d76-4af5-adcf-ecf54a4e09ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:33:36.343235 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a18daf8-5d76-4af5-adcf-ecf54a4e09ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:33:36.344231 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a18daf8-5d76-4af5-adcf-ecf54a4e09ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:33:36.344509 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a18daf8-5d76-4af5-adcf-ecf54a4e09ed admin admin] PATH_INFO: `/v3/projects/f6a0ed4c4c17430fab1dadfc5516fab9` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:33:36.345861 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a18daf8-5d76-4af5-adcf-ecf54a4e09ed admin admin] RBAC: Authorizing `identity:delete_project(project_id=f6a0ed4c4c17430fab1dadfc5516fab9)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:33:36.347851 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a18daf8-5d76-4af5-adcf-ecf54a4e09ed admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:33:36.463420 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-4a18daf8-5d76-4af5-adcf-ecf54a4e09ed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f6a0ed4c4c17430fab1dadfc5516fab9', 'request_id': 'req-4a18daf8-5d76-4af5-adcf-ecf54a4e09ed'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:33:36.470360 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-4a18daf8-5d76-4af5-adcf-ecf54a4e09ed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f6a0ed4c4c17430fab1dadfc5516fab9', 'request_id': 'req-4a18daf8-5d76-4af5-adcf-ecf54a4e09ed'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:33:36.477063 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-4a18daf8-5d76-4af5-adcf-ecf54a4e09ed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f6a0ed4c4c17430fab1dadfc5516fab9', 'request_id': 'req-4a18daf8-5d76-4af5-adcf-ecf54a4e09ed'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:33:36.547519 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-4a18daf8-5d76-4af5-adcf-ecf54a4e09ed admin admin] The token cache is being invalidate because project f6a0ed4c4c17430fab1dadfc5516fab9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81994) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 15:33:36.547810 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-4a18daf8-5d76-4af5-adcf-ecf54a4e09ed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:33:36.548457 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-4a18daf8-5d76-4af5-adcf-ecf54a4e09ed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:33:36.549923 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1439/2855] 10.222.0.55 () {58 vars in 1233 bytes} [Fri Sep 23 15:33:36 2022] DELETE /identity/v3/projects/f6a0ed4c4c17430fab1dadfc5516fab9 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:33:36.554985 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314f8d3d-4c90-463e-bb87-a505077f726e None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:33:36.555500 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314f8d3d-4c90-463e-bb87-a505077f726e None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:33:36.555630 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314f8d3d-4c90-463e-bb87-a505077f726e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:33:36.575489 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-314f8d3d-4c90-463e-bb87-a505077f726e None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:33:36.576654 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-314f8d3d-4c90-463e-bb87-a505077f726e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:33:36.638605 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1417/2856] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:33:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:33:36.647317 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e4625d-166b-4833-92a2-a8cfb488a061 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:33:36.647836 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e4625d-166b-4833-92a2-a8cfb488a061 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:33:36.647960 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e4625d-166b-4833-92a2-a8cfb488a061 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:33:36.668291 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-50e4625d-166b-4833-92a2-a8cfb488a061 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:33:36.669547 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-50e4625d-166b-4833-92a2-a8cfb488a061 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:33:36.678153 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1440/2857] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:33:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:33:36.682466 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314f8d3d-4c90-463e-bb87-a505077f726e None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:33:36.693284 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314f8d3d-4c90-463e-bb87-a505077f726e None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:33:36.698134 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6229ef7a-2fa7-4d9d-8166-435d8419ae48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:33:36.698745 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6229ef7a-2fa7-4d9d-8166-435d8419ae48 admin admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:33:36.699028 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6229ef7a-2fa7-4d9d-8166-435d8419ae48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:33:36.699304 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6229ef7a-2fa7-4d9d-8166-435d8419ae48 admin admin] PATH_INFO: `/v3/domains` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:33:36.699958 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6229ef7a-2fa7-4d9d-8166-435d8419ae48 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81993) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:33:36.700298 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6229ef7a-2fa7-4d9d-8166-435d8419ae48 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:33:36.702051 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6229ef7a-2fa7-4d9d-8166-435d8419ae48 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:33:36.714904 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1418/2858] 10.222.0.55 () {58 vars in 1153 bytes} [Fri Sep 23 15:33:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:33:36.719553 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e4625d-166b-4833-92a2-a8cfb488a061 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:33:36.729781 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e4625d-166b-4833-92a2-a8cfb488a061 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:33:36.734620 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591660e8-f91a-46de-a5b6-9647bd52f26b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:33:36.735336 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591660e8-f91a-46de-a5b6-9647bd52f26b admin admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:33:36.735649 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591660e8-f91a-46de-a5b6-9647bd52f26b admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:33:36.736004 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591660e8-f91a-46de-a5b6-9647bd52f26b admin admin] PATH_INFO: `/v3/projects` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:33:36.736653 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591660e8-f91a-46de-a5b6-9647bd52f26b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:33:36.738801 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591660e8-f91a-46de-a5b6-9647bd52f26b admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:33:36.842404 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1441/2859] 10.222.0.55 () {60 vars in 1153 bytes} [Fri Sep 23 15:33:36 2022] POST /identity/v3/projects => generated 381 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:33:36.845885 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6229ef7a-2fa7-4d9d-8166-435d8419ae48 admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:33:36.857375 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6229ef7a-2fa7-4d9d-8166-435d8419ae48 admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:33:36.904970 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:33:36.918460 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18938f2f-6afb-4c93-8c5c-8cc044faeb41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:33:36.919060 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18938f2f-6afb-4c93-8c5c-8cc044faeb41 admin admin] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:33:36.919335 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18938f2f-6afb-4c93-8c5c-8cc044faeb41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:33:36.919603 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18938f2f-6afb-4c93-8c5c-8cc044faeb41 admin admin] PATH_INFO: `/v3/users` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:33:36.920275 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18938f2f-6afb-4c93-8c5c-8cc044faeb41 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:33:36.922186 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18938f2f-6afb-4c93-8c5c-8cc044faeb41 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:33:37.005892 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1419/2860] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Sep 23 15:33:36 2022] POST /identity/v3/users => generated 291 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:33:37.010072 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591660e8-f91a-46de-a5b6-9647bd52f26b admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:33:37.021747 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591660e8-f91a-46de-a5b6-9647bd52f26b admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:33:37.026577 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcf4053-2701-420e-a04c-a012410dedeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:33:37.027196 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcf4053-2701-420e-a04c-a012410dedeb admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:33:37.027470 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcf4053-2701-420e-a04c-a012410dedeb admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:33:37.027741 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcf4053-2701-420e-a04c-a012410dedeb admin admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:33:37.028423 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcf4053-2701-420e-a04c-a012410dedeb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:33:37.030006 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcf4053-2701-420e-a04c-a012410dedeb admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:33:37.041022 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1442/2861] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Sep 23 15:33:37 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:33:37.045115 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18938f2f-6afb-4c93-8c5c-8cc044faeb41 admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:33:37.055340 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18938f2f-6afb-4c93-8c5c-8cc044faeb41 admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:33:37.059960 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88dce86a-ac98-4065-9d76-b7415df81fa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:33:37.060586 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dce86a-ac98-4065-9d76-b7415df81fa8 admin admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:33:37.060867 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dce86a-ac98-4065-9d76-b7415df81fa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:33:37.061143 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dce86a-ac98-4065-9d76-b7415df81fa8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:33:37.061777 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88dce86a-ac98-4065-9d76-b7415df81fa8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:33:37.063378 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88dce86a-ac98-4065-9d76-b7415df81fa8 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:33:37.074615 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1420/2862] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Sep 23 15:33:37 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:33:37.079067 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcf4053-2701-420e-a04c-a012410dedeb admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:33:37.090547 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcf4053-2701-420e-a04c-a012410dedeb admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:33:37.095342 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc326a98-fcd2-49b4-9022-1950a7d34310 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:33:37.095940 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc326a98-fcd2-49b4-9022-1950a7d34310 admin admin] REQUEST_METHOD: `PUT` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:33:37.096256 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc326a98-fcd2-49b4-9022-1950a7d34310 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:33:37.096532 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc326a98-fcd2-49b4-9022-1950a7d34310 admin admin] PATH_INFO: `/v3/projects/91a602d5b45b4e808cb18341f4cddcfe/users/45c0e65f397643ce9b9afe9014eb599f/roles/af257be2dee54d6b942f58650196582c` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:33:37.102062 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:33:37.114993 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc326a98-fcd2-49b4-9022-1950a7d34310 admin admin] RBAC: Authorizing `identity:create_grant(project_id=91a602d5b45b4e808cb18341f4cddcfe, user_id=45c0e65f397643ce9b9afe9014eb599f, role_id=af257be2dee54d6b942f58650196582c)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:33:37.119970 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc326a98-fcd2-49b4-9022-1950a7d34310 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:33:37.229115 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1443/2863] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Sep 23 15:33:37 2022] PUT /identity/v3/projects/91a602d5b45b4e808cb18341f4cddcfe/users/45c0e65f397643ce9b9afe9014eb599f/roles/af257be2dee54d6b942f58650196582c => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:33:37.241236 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a962f6ef-8ec3-4740-8d25-7d36566a357f None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:33:37.241520 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a962f6ef-8ec3-4740-8d25-7d36566a357f None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:33:37.241797 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a962f6ef-8ec3-4740-8d25-7d36566a357f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:33:37.271048 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-a962f6ef-8ec3-4740-8d25-7d36566a357f None None] MFA Rules not processed for user `45c0e65f397643ce9b9afe9014eb599f`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:33:37.272372 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-a962f6ef-8ec3-4740-8d25-7d36566a357f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:33:37.339235 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1421/2864] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:33:37 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 15:33:37.343949 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc326a98-fcd2-49b4-9022-1950a7d34310 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:33:37.408231 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc326a98-fcd2-49b4-9022-1950a7d34310 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:33:37.413032 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebce6ca6-ecc9-4ed8-bd17-f5fd3595b38c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:33:37.413643 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebce6ca6-ecc9-4ed8-bd17-f5fd3595b38c admin admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:33:37.413928 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebce6ca6-ecc9-4ed8-bd17-f5fd3595b38c admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:33:37.414205 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebce6ca6-ecc9-4ed8-bd17-f5fd3595b38c admin admin] PATH_INFO: `/v3/projects` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:33:37.414853 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebce6ca6-ecc9-4ed8-bd17-f5fd3595b38c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:33:37.416793 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebce6ca6-ecc9-4ed8-bd17-f5fd3595b38c admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:33:37.538862 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1444/2865] 10.222.0.55 () {60 vars in 1153 bytes} [Fri Sep 23 15:33:37 2022] POST /identity/v3/projects => generated 385 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:33:37.543171 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a962f6ef-8ec3-4740-8d25-7d36566a357f None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:33:37.555691 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a962f6ef-8ec3-4740-8d25-7d36566a357f None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:33:37.617710 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c73ad2-e713-4c60-b230-75cfc5f0c8a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:33:37.618329 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c73ad2-e713-4c60-b230-75cfc5f0c8a6 admin admin] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:33:37.618615 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c73ad2-e713-4c60-b230-75cfc5f0c8a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:33:37.618906 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c73ad2-e713-4c60-b230-75cfc5f0c8a6 admin admin] PATH_INFO: `/v3/users` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:33:37.619551 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c73ad2-e713-4c60-b230-75cfc5f0c8a6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:33:37.621532 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c73ad2-e713-4c60-b230-75cfc5f0c8a6 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:33:37.771892 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1422/2866] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Sep 23 15:33:37 2022] POST /identity/v3/users => generated 293 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:33:37.775970 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebce6ca6-ecc9-4ed8-bd17-f5fd3595b38c admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:33:37.787866 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebce6ca6-ecc9-4ed8-bd17-f5fd3595b38c admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:33:37.792511 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b91beaf-dd4a-4798-9f26-ad4f6a95a89e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:33:37.793117 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b91beaf-dd4a-4798-9f26-ad4f6a95a89e admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:33:37.793401 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b91beaf-dd4a-4798-9f26-ad4f6a95a89e admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:33:37.793794 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b91beaf-dd4a-4798-9f26-ad4f6a95a89e admin admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:33:37.794440 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b91beaf-dd4a-4798-9f26-ad4f6a95a89e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:33:37.796073 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b91beaf-dd4a-4798-9f26-ad4f6a95a89e admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:33:37.808150 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1445/2867] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Sep 23 15:33:37 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:33:37.812237 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c73ad2-e713-4c60-b230-75cfc5f0c8a6 admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:33:37.822381 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c73ad2-e713-4c60-b230-75cfc5f0c8a6 admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:33:37.827032 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec5cde5-ab17-4472-ba84-19d074b72b9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:33:37.827630 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec5cde5-ab17-4472-ba84-19d074b72b9d admin admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:33:37.827907 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec5cde5-ab17-4472-ba84-19d074b72b9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:33:37.828238 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec5cde5-ab17-4472-ba84-19d074b72b9d admin admin] PATH_INFO: `/v3/roles` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:33:37.828878 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec5cde5-ab17-4472-ba84-19d074b72b9d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:33:37.830462 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec5cde5-ab17-4472-ba84-19d074b72b9d admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:33:37.841069 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1423/2868] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Sep 23 15:33:37 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:33:37.845302 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b91beaf-dd4a-4798-9f26-ad4f6a95a89e admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:33:37.856105 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b91beaf-dd4a-4798-9f26-ad4f6a95a89e admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:33:37.860721 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c104364-9c83-4838-87c1-12e748857c26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:33:37.861376 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c104364-9c83-4838-87c1-12e748857c26 admin admin] REQUEST_METHOD: `PUT` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:33:37.861653 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c104364-9c83-4838-87c1-12e748857c26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:33:37.862188 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c104364-9c83-4838-87c1-12e748857c26 admin admin] PATH_INFO: `/v3/projects/9247e0a68e744b76a1c800c5e2112a00/users/8213af003101480f96bfa4f146b571a6/roles/af257be2dee54d6b942f58650196582c` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:33:37.877740 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c104364-9c83-4838-87c1-12e748857c26 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9247e0a68e744b76a1c800c5e2112a00, user_id=8213af003101480f96bfa4f146b571a6, role_id=af257be2dee54d6b942f58650196582c)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:33:37.880807 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c104364-9c83-4838-87c1-12e748857c26 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:33:37.947872 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1446/2869] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Sep 23 15:33:37 2022] PUT /identity/v3/projects/9247e0a68e744b76a1c800c5e2112a00/users/8213af003101480f96bfa4f146b571a6/roles/af257be2dee54d6b942f58650196582c => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:33:37.959034 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7d1764-c622-46c6-8743-11e2eafcb5e3 None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:33:37.959322 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7d1764-c622-46c6-8743-11e2eafcb5e3 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:33:37.959629 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7d1764-c622-46c6-8743-11e2eafcb5e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:33:37.989732 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-be7d1764-c622-46c6-8743-11e2eafcb5e3 None None] MFA Rules not processed for user `8213af003101480f96bfa4f146b571a6`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:33:37.990957 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-be7d1764-c622-46c6-8743-11e2eafcb5e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:33:38.011011 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:33:38.057780 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1424/2870] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:33:37 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:33:38.062269 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c104364-9c83-4838-87c1-12e748857c26 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:33:38.126448 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c104364-9c83-4838-87c1-12e748857c26 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:33:38.132179 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aeca8bb-5aec-4bda-a473-87b1a5a2312c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:33:38.132790 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aeca8bb-5aec-4bda-a473-87b1a5a2312c admin admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:33:38.133070 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aeca8bb-5aec-4bda-a473-87b1a5a2312c admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:33:38.133341 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aeca8bb-5aec-4bda-a473-87b1a5a2312c admin admin] PATH_INFO: `/v3/projects` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:33:38.133985 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aeca8bb-5aec-4bda-a473-87b1a5a2312c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:33:38.135864 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aeca8bb-5aec-4bda-a473-87b1a5a2312c admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:33:38.258850 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1447/2871] 10.222.0.55 () {60 vars in 1153 bytes} [Fri Sep 23 15:33:38 2022] POST /identity/v3/projects => generated 383 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:33:38.263175 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7d1764-c622-46c6-8743-11e2eafcb5e3 None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:33:38.275956 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7d1764-c622-46c6-8743-11e2eafcb5e3 None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:33:38.337314 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3d2dbf-8ca1-4760-8b54-76c5e4404aff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:33:38.337843 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3d2dbf-8ca1-4760-8b54-76c5e4404aff admin admin] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:33:38.338126 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3d2dbf-8ca1-4760-8b54-76c5e4404aff admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:33:38.338404 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3d2dbf-8ca1-4760-8b54-76c5e4404aff admin admin] PATH_INFO: `/v3/users` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:33:38.339030 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3d2dbf-8ca1-4760-8b54-76c5e4404aff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:33:38.341003 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3d2dbf-8ca1-4760-8b54-76c5e4404aff admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:33:38.463134 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1425/2872] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Sep 23 15:33:38 2022] POST /identity/v3/users => generated 298 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:33:38.467305 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aeca8bb-5aec-4bda-a473-87b1a5a2312c admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:33:38.478762 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aeca8bb-5aec-4bda-a473-87b1a5a2312c admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:33:38.483420 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd663b8-80de-41cf-bc65-8380db2dbb3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:33:38.484054 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd663b8-80de-41cf-bc65-8380db2dbb3c admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:33:38.484339 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd663b8-80de-41cf-bc65-8380db2dbb3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:33:38.484613 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd663b8-80de-41cf-bc65-8380db2dbb3c admin admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:33:38.485250 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd663b8-80de-41cf-bc65-8380db2dbb3c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:33:38.487116 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd663b8-80de-41cf-bc65-8380db2dbb3c admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:33:38.498454 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1448/2873] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Sep 23 15:33:38 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 23 15:33:38.503194 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3d2dbf-8ca1-4760-8b54-76c5e4404aff admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:33:38.514798 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3d2dbf-8ca1-4760-8b54-76c5e4404aff admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:33:38.519360 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b448164-2f53-455f-940d-c845c4cf66e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:33:38.520051 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b448164-2f53-455f-940d-c845c4cf66e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:33:38.520334 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b448164-2f53-455f-940d-c845c4cf66e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:33:38.520609 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b448164-2f53-455f-940d-c845c4cf66e9 admin admin] PATH_INFO: `/v3/projects/4cefed444f7f49aabf3a04180f4d462d/users/97a6cf848a7e4e5b96ba07a209b3793b/roles/8e2919e79df040e8926f21607818b58c` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:33:38.536130 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b448164-2f53-455f-940d-c845c4cf66e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4cefed444f7f49aabf3a04180f4d462d, user_id=97a6cf848a7e4e5b96ba07a209b3793b, role_id=8e2919e79df040e8926f21607818b58c)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:33:38.539566 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b448164-2f53-455f-940d-c845c4cf66e9 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:33:38.651111 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1426/2874] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Sep 23 15:33:38 2022] PUT /identity/v3/projects/4cefed444f7f49aabf3a04180f4d462d/users/97a6cf848a7e4e5b96ba07a209b3793b/roles/8e2919e79df040e8926f21607818b58c => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:33:38.655089 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd663b8-80de-41cf-bc65-8380db2dbb3c admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:33:38.665756 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd663b8-80de-41cf-bc65-8380db2dbb3c admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:33:38.672988 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:33:38.723179 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7e98dd-550c-42dc-bcc6-4b6d2e7a9d03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:33:38.723789 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7e98dd-550c-42dc-bcc6-4b6d2e7a9d03 admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:33:38.724129 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7e98dd-550c-42dc-bcc6-4b6d2e7a9d03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:33:38.724409 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7e98dd-550c-42dc-bcc6-4b6d2e7a9d03 admin admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:33:38.725208 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7e98dd-550c-42dc-bcc6-4b6d2e7a9d03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:33:38.726790 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7e98dd-550c-42dc-bcc6-4b6d2e7a9d03 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:33:38.736768 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1449/2875] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Sep 23 15:33:38 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:33:38.741843 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b448164-2f53-455f-940d-c845c4cf66e9 admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:33:38.752172 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b448164-2f53-455f-940d-c845c4cf66e9 admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:33:38.757465 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504d6659-f488-4d85-b393-07fd669d9708 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:33:38.758959 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504d6659-f488-4d85-b393-07fd669d9708 admin admin] REQUEST_METHOD: `PUT` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:33:38.758959 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504d6659-f488-4d85-b393-07fd669d9708 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:33:38.758959 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504d6659-f488-4d85-b393-07fd669d9708 admin admin] PATH_INFO: `/v3/projects/4cefed444f7f49aabf3a04180f4d462d/users/97a6cf848a7e4e5b96ba07a209b3793b/roles/8e2919e79df040e8926f21607818b58c` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:33:38.760648 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504d6659-f488-4d85-b393-07fd669d9708 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4cefed444f7f49aabf3a04180f4d462d, user_id=97a6cf848a7e4e5b96ba07a209b3793b, role_id=8e2919e79df040e8926f21607818b58c)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:33:38.763252 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504d6659-f488-4d85-b393-07fd669d9708 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:33:38.773922 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1427/2876] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Sep 23 15:33:38 2022] PUT /identity/v3/projects/4cefed444f7f49aabf3a04180f4d462d/users/97a6cf848a7e4e5b96ba07a209b3793b/roles/8e2919e79df040e8926f21607818b58c => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:33:38.785367 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97323902-6d06-46ff-9a2a-36786e83b0fc None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:33:38.785926 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97323902-6d06-46ff-9a2a-36786e83b0fc None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:33:38.785926 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97323902-6d06-46ff-9a2a-36786e83b0fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:33:38.815386 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-97323902-6d06-46ff-9a2a-36786e83b0fc None None] MFA Rules not processed for user `97a6cf848a7e4e5b96ba07a209b3793b`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:33:38.816662 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-97323902-6d06-46ff-9a2a-36786e83b0fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:33:38.888305 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1450/2877] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:33:38 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 15:33:38.901092 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504d6659-f488-4d85-b393-07fd669d9708 admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:33:38.902892 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-504d6659-f488-4d85-b393-07fd669d9708 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:33:38.965961 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504d6659-f488-4d85-b393-07fd669d9708 admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:33:38.969919 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8825969-1700-4140-9544-48df75ae8034 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feca6278be904ef8b0dc580a67e477f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:33:38.970551 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8825969-1700-4140-9544-48df75ae8034 service cinder] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:33:38.970828 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8825969-1700-4140-9544-48df75ae8034 service cinder] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:33:38.971095 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8825969-1700-4140-9544-48df75ae8034 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:33:39.003488 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:33:39.031779 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8825969-1700-4140-9544-48df75ae8034 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:33:39.033273 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8825969-1700-4140-9544-48df75ae8034 service cinder] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:33:39.042956 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1428/2878] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:33:38 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:33:41.135821 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97323902-6d06-46ff-9a2a-36786e83b0fc None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:33:41.137274 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-97323902-6d06-46ff-9a2a-36786e83b0fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:33:41.151124 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97323902-6d06-46ff-9a2a-36786e83b0fc None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:33:41.156485 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469aaae9-61e0-473b-880d-8c95bc79be92 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feca6278be904ef8b0dc580a67e477f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:33:41.156485 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469aaae9-61e0-473b-880d-8c95bc79be92 service cinder] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:33:41.156485 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469aaae9-61e0-473b-880d-8c95bc79be92 service cinder] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:33:41.156485 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469aaae9-61e0-473b-880d-8c95bc79be92 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:33:41.216422 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469aaae9-61e0-473b-880d-8c95bc79be92 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:33:41.217974 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469aaae9-61e0-473b-880d-8c95bc79be92 service cinder] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:33:41.228440 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1451/2879] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:33:41 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:33:42.811765 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8825969-1700-4140-9544-48df75ae8034 service cinder] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:33:42.820637 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8825969-1700-4140-9544-48df75ae8034 service cinder] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:33:42.824218 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a72e73-f2ca-4ffd-9513-c327176b4220 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feca6278be904ef8b0dc580a67e477f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:33:42.824812 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a72e73-f2ca-4ffd-9513-c327176b4220 service cinder] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:33:42.825090 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a72e73-f2ca-4ffd-9513-c327176b4220 service cinder] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:33:42.825356 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a72e73-f2ca-4ffd-9513-c327176b4220 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:33:42.834199 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a72e73-f2ca-4ffd-9513-c327176b4220 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:33:42.836177 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a72e73-f2ca-4ffd-9513-c327176b4220 service cinder] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:33:42.845542 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1429/2880] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:33:42 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:33:47.545637 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469aaae9-61e0-473b-880d-8c95bc79be92 service cinder] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:33:47.578392 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=100 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:33:47.612488 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469aaae9-61e0-473b-880d-8c95bc79be92 service cinder] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:33:47.617741 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb99dd17-c362-4cd3-9ce9-d14441c72299 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:33:47.618377 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb99dd17-c362-4cd3-9ce9-d14441c72299 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:33:47.618664 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb99dd17-c362-4cd3-9ce9-d14441c72299 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:33:47.618942 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb99dd17-c362-4cd3-9ce9-d14441c72299 admin admin] PATH_INFO: `/v3/users/45c0e65f397643ce9b9afe9014eb599f` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:33:47.620110 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb99dd17-c362-4cd3-9ce9-d14441c72299 admin admin] RBAC: Authorizing `identity:delete_user(user_id=45c0e65f397643ce9b9afe9014eb599f)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:33:47.622040 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb99dd17-c362-4cd3-9ce9-d14441c72299 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:33:47.836475 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-eb99dd17-c362-4cd3-9ce9-d14441c72299 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '45c0e65f397643ce9b9afe9014eb599f'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:33:47.851603 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-eb99dd17-c362-4cd3-9ce9-d14441c72299 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '45c0e65f397643ce9b9afe9014eb599f'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:33:47.858970 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-eb99dd17-c362-4cd3-9ce9-d14441c72299 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '45c0e65f397643ce9b9afe9014eb599f'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:33:47.924518 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-eb99dd17-c362-4cd3-9ce9-d14441c72299 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '45c0e65f397643ce9b9afe9014eb599f'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:33:47.925158 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-eb99dd17-c362-4cd3-9ce9-d14441c72299 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '45c0e65f397643ce9b9afe9014eb599f'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:33:47.932431 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1452/2881] 10.222.0.55 () {58 vars in 1224 bytes} [Fri Sep 23 15:33:47 2022] DELETE /identity/v3/users/45c0e65f397643ce9b9afe9014eb599f => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:33:47.937097 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1fce09-e6ed-419b-b806-d82b186fd748 None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:33:47.937699 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1fce09-e6ed-419b-b806-d82b186fd748 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:33:47.937699 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1fce09-e6ed-419b-b806-d82b186fd748 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:33:47.957024 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-5f1fce09-e6ed-419b-b806-d82b186fd748 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:33:47.958271 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-5f1fce09-e6ed-419b-b806-d82b186fd748 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:33:48.025026 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1430/2882] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:33:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:33:48.035145 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb99dd17-c362-4cd3-9ce9-d14441c72299 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:33:48.036677 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-eb99dd17-c362-4cd3-9ce9-d14441c72299 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:33:48.116173 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb99dd17-c362-4cd3-9ce9-d14441c72299 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:33:48.122275 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be49679-54e1-42cb-8565-9c33ee781aed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd259178570864ffeb936ee10dfec6b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:33:48.123313 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be49679-54e1-42cb-8565-9c33ee781aed service neutron] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:33:48.123817 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be49679-54e1-42cb-8565-9c33ee781aed service neutron] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:33:48.124355 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be49679-54e1-42cb-8565-9c33ee781aed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:33:48.139335 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be49679-54e1-42cb-8565-9c33ee781aed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:33:48.140808 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be49679-54e1-42cb-8565-9c33ee781aed service neutron] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:33:48.151850 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1453/2883] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:33:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:33:48.515480 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1fce09-e6ed-419b-b806-d82b186fd748 None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:33:48.516938 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-5f1fce09-e6ed-419b-b806-d82b186fd748 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:33:48.530172 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1fce09-e6ed-419b-b806-d82b186fd748 None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:33:48.534745 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14a76e8-53a3-4e8b-b6ce-0693608bfd0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:33:48.535392 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14a76e8-53a3-4e8b-b6ce-0693608bfd0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:33:48.535667 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14a76e8-53a3-4e8b-b6ce-0693608bfd0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:33:48.535934 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14a76e8-53a3-4e8b-b6ce-0693608bfd0d admin admin] PATH_INFO: `/v3/projects/91a602d5b45b4e808cb18341f4cddcfe` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:33:48.537319 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14a76e8-53a3-4e8b-b6ce-0693608bfd0d admin admin] RBAC: Authorizing `identity:delete_project(project_id=91a602d5b45b4e808cb18341f4cddcfe)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:33:48.539271 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14a76e8-53a3-4e8b-b6ce-0693608bfd0d admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:33:48.646104 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-c14a76e8-53a3-4e8b-b6ce-0693608bfd0d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '91a602d5b45b4e808cb18341f4cddcfe', 'request_id': 'req-c14a76e8-53a3-4e8b-b6ce-0693608bfd0d'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:33:48.652660 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-c14a76e8-53a3-4e8b-b6ce-0693608bfd0d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '91a602d5b45b4e808cb18341f4cddcfe', 'request_id': 'req-c14a76e8-53a3-4e8b-b6ce-0693608bfd0d'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:33:48.856409 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-c14a76e8-53a3-4e8b-b6ce-0693608bfd0d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '91a602d5b45b4e808cb18341f4cddcfe', 'request_id': 'req-c14a76e8-53a3-4e8b-b6ce-0693608bfd0d'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:33:48.869008 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-c14a76e8-53a3-4e8b-b6ce-0693608bfd0d admin admin] The token cache is being invalidate because project 91a602d5b45b4e808cb18341f4cddcfe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81993) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 15:33:48.870373 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-c14a76e8-53a3-4e8b-b6ce-0693608bfd0d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:33:48.870373 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-c14a76e8-53a3-4e8b-b6ce-0693608bfd0d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:33:48.871470 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1431/2884] 10.222.0.55 () {58 vars in 1233 bytes} [Fri Sep 23 15:33:48 2022] DELETE /identity/v3/projects/91a602d5b45b4e808cb18341f4cddcfe => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:33:48.875459 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be49679-54e1-42cb-8565-9c33ee781aed service neutron] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:33:48.887457 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be49679-54e1-42cb-8565-9c33ee781aed service neutron] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:33:48.938297 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:33:48.949190 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9eba255-6e3d-4f12-bdec-2cd03eab31cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:33:48.949830 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9eba255-6e3d-4f12-bdec-2cd03eab31cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:33:48.950112 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9eba255-6e3d-4f12-bdec-2cd03eab31cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:33:48.950389 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9eba255-6e3d-4f12-bdec-2cd03eab31cb admin admin] PATH_INFO: `/v3/users/8213af003101480f96bfa4f146b571a6` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:33:48.951509 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9eba255-6e3d-4f12-bdec-2cd03eab31cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=8213af003101480f96bfa4f146b571a6)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:33:48.953462 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9eba255-6e3d-4f12-bdec-2cd03eab31cb admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:33:49.190288 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-d9eba255-6e3d-4f12-bdec-2cd03eab31cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8213af003101480f96bfa4f146b571a6'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:33:49.206995 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-d9eba255-6e3d-4f12-bdec-2cd03eab31cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8213af003101480f96bfa4f146b571a6'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:33:49.213892 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-d9eba255-6e3d-4f12-bdec-2cd03eab31cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8213af003101480f96bfa4f146b571a6'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:33:49.278639 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-d9eba255-6e3d-4f12-bdec-2cd03eab31cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8213af003101480f96bfa4f146b571a6'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:33:49.279281 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-d9eba255-6e3d-4f12-bdec-2cd03eab31cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8213af003101480f96bfa4f146b571a6'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:33:49.285660 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1454/2885] 10.222.0.55 () {58 vars in 1224 bytes} [Fri Sep 23 15:33:48 2022] DELETE /identity/v3/users/8213af003101480f96bfa4f146b571a6 => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:33:49.745403 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14a76e8-53a3-4e8b-b6ce-0693608bfd0d admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:33:49.763123 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-c14a76e8-53a3-4e8b-b6ce-0693608bfd0d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:33:49.763123 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14a76e8-53a3-4e8b-b6ce-0693608bfd0d admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:33:49.806447 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:33:49.824904 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc47415a-5679-4102-aa8f-0c0a48d752f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:33:49.825534 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc47415a-5679-4102-aa8f-0c0a48d752f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:33:49.825815 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc47415a-5679-4102-aa8f-0c0a48d752f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:33:49.826083 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc47415a-5679-4102-aa8f-0c0a48d752f6 admin admin] PATH_INFO: `/v3/projects/9247e0a68e744b76a1c800c5e2112a00` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:33:49.827192 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc47415a-5679-4102-aa8f-0c0a48d752f6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9247e0a68e744b76a1c800c5e2112a00)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:33:49.829113 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc47415a-5679-4102-aa8f-0c0a48d752f6 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:33:49.936350 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-cc47415a-5679-4102-aa8f-0c0a48d752f6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9247e0a68e744b76a1c800c5e2112a00', 'request_id': 'req-cc47415a-5679-4102-aa8f-0c0a48d752f6'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:33:49.941745 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-cc47415a-5679-4102-aa8f-0c0a48d752f6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9247e0a68e744b76a1c800c5e2112a00', 'request_id': 'req-cc47415a-5679-4102-aa8f-0c0a48d752f6'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:33:50.026195 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-cc47415a-5679-4102-aa8f-0c0a48d752f6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9247e0a68e744b76a1c800c5e2112a00', 'request_id': 'req-cc47415a-5679-4102-aa8f-0c0a48d752f6'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:33:50.038408 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-cc47415a-5679-4102-aa8f-0c0a48d752f6 admin admin] The token cache is being invalidate because project 9247e0a68e744b76a1c800c5e2112a00 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81993) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 15:33:50.038695 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-cc47415a-5679-4102-aa8f-0c0a48d752f6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:33:50.039375 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-cc47415a-5679-4102-aa8f-0c0a48d752f6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:33:50.040997 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1432/2886] 10.222.0.55 () {58 vars in 1233 bytes} [Fri Sep 23 15:33:49 2022] DELETE /identity/v3/projects/9247e0a68e744b76a1c800c5e2112a00 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:33:50.046143 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9eba255-6e3d-4f12-bdec-2cd03eab31cb admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:33:50.047585 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-d9eba255-6e3d-4f12-bdec-2cd03eab31cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:33:50.061010 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9eba255-6e3d-4f12-bdec-2cd03eab31cb admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:33:50.120891 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6e1aa0-8caf-41f4-ac1a-6a28c8e0e627 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:33:50.121543 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6e1aa0-8caf-41f4-ac1a-6a28c8e0e627 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:33:50.121825 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6e1aa0-8caf-41f4-ac1a-6a28c8e0e627 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:33:50.122110 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6e1aa0-8caf-41f4-ac1a-6a28c8e0e627 admin admin] PATH_INFO: `/v3/users/97a6cf848a7e4e5b96ba07a209b3793b` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:33:50.123259 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6e1aa0-8caf-41f4-ac1a-6a28c8e0e627 admin admin] RBAC: Authorizing `identity:delete_user(user_id=97a6cf848a7e4e5b96ba07a209b3793b)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:33:50.125205 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6e1aa0-8caf-41f4-ac1a-6a28c8e0e627 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:33:50.247413 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-4d6e1aa0-8caf-41f4-ac1a-6a28c8e0e627 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '97a6cf848a7e4e5b96ba07a209b3793b'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:33:50.264615 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-4d6e1aa0-8caf-41f4-ac1a-6a28c8e0e627 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '97a6cf848a7e4e5b96ba07a209b3793b'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:33:50.270138 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=80,keystone:DELETE=19,keystone:INSERT=1 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:33:50.272769 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-4d6e1aa0-8caf-41f4-ac1a-6a28c8e0e627 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '97a6cf848a7e4e5b96ba07a209b3793b'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:33:50.336655 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-4d6e1aa0-8caf-41f4-ac1a-6a28c8e0e627 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '97a6cf848a7e4e5b96ba07a209b3793b'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:33:50.337292 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-4d6e1aa0-8caf-41f4-ac1a-6a28c8e0e627 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '97a6cf848a7e4e5b96ba07a209b3793b'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:33:50.343914 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1455/2887] 10.222.0.55 () {58 vars in 1224 bytes} [Fri Sep 23 15:33:50 2022] DELETE /identity/v3/users/97a6cf848a7e4e5b96ba07a209b3793b => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:33:50.761214 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc47415a-5679-4102-aa8f-0c0a48d752f6 admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:33:50.765281 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-cc47415a-5679-4102-aa8f-0c0a48d752f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:33:50.778070 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc47415a-5679-4102-aa8f-0c0a48d752f6 admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:33:50.838663 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc681d5-6925-45c5-9a04-f0193efba82f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:33:50.839215 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc681d5-6925-45c5-9a04-f0193efba82f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:33:50.839494 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc681d5-6925-45c5-9a04-f0193efba82f admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:33:50.839768 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc681d5-6925-45c5-9a04-f0193efba82f admin admin] PATH_INFO: `/v3/projects/4cefed444f7f49aabf3a04180f4d462d` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:33:50.840929 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc681d5-6925-45c5-9a04-f0193efba82f admin admin] RBAC: Authorizing `identity:delete_project(project_id=4cefed444f7f49aabf3a04180f4d462d)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:33:50.842833 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc681d5-6925-45c5-9a04-f0193efba82f admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:33:50.988803 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-6dc681d5-6925-45c5-9a04-f0193efba82f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4cefed444f7f49aabf3a04180f4d462d', 'request_id': 'req-6dc681d5-6925-45c5-9a04-f0193efba82f'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:33:50.995275 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-6dc681d5-6925-45c5-9a04-f0193efba82f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4cefed444f7f49aabf3a04180f4d462d', 'request_id': 'req-6dc681d5-6925-45c5-9a04-f0193efba82f'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:33:51.078775 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-6dc681d5-6925-45c5-9a04-f0193efba82f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4cefed444f7f49aabf3a04180f4d462d', 'request_id': 'req-6dc681d5-6925-45c5-9a04-f0193efba82f'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:33:51.087535 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:33:51.091553 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-6dc681d5-6925-45c5-9a04-f0193efba82f admin admin] The token cache is being invalidate because project 4cefed444f7f49aabf3a04180f4d462d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81993) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 15:33:51.091786 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-6dc681d5-6925-45c5-9a04-f0193efba82f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:33:51.092568 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-6dc681d5-6925-45c5-9a04-f0193efba82f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:33:51.094077 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1433/2888] 10.222.0.55 () {58 vars in 1233 bytes} [Fri Sep 23 15:33:50 2022] DELETE /identity/v3/projects/4cefed444f7f49aabf3a04180f4d462d => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:33:51.099085 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509cd7b7-9c35-4fed-bab0-8373763de018 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:33:51.100117 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509cd7b7-9c35-4fed-bab0-8373763de018 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:33:51.100270 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509cd7b7-9c35-4fed-bab0-8373763de018 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:33:51.120987 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-509cd7b7-9c35-4fed-bab0-8373763de018 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:33:51.122196 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-509cd7b7-9c35-4fed-bab0-8373763de018 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:33:51.182468 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1456/2889] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:33:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:33:51.190521 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f30660-73f9-4ee8-bc0e-dce575ffd022 None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:33:51.191914 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f30660-73f9-4ee8-bc0e-dce575ffd022 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:33:51.191914 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f30660-73f9-4ee8-bc0e-dce575ffd022 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:33:51.211839 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-33f30660-73f9-4ee8-bc0e-dce575ffd022 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:33:51.213073 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-33f30660-73f9-4ee8-bc0e-dce575ffd022 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:33:51.221520 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1434/2890] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:33:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:33:51.226067 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509cd7b7-9c35-4fed-bab0-8373763de018 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:33:51.235955 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509cd7b7-9c35-4fed-bab0-8373763de018 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:33:51.241190 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd4da4a-8d49-44b6-86c5-48e5781fe091 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:33:51.241802 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd4da4a-8d49-44b6-86c5-48e5781fe091 admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:33:51.242088 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd4da4a-8d49-44b6-86c5-48e5781fe091 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:33:51.242371 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd4da4a-8d49-44b6-86c5-48e5781fe091 admin admin] PATH_INFO: `/v3/domains` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:33:51.243115 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd4da4a-8d49-44b6-86c5-48e5781fe091 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:33:51.243423 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd4da4a-8d49-44b6-86c5-48e5781fe091 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:33:51.245055 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd4da4a-8d49-44b6-86c5-48e5781fe091 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:33:51.257915 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1457/2891] 10.222.0.55 () {58 vars in 1153 bytes} [Fri Sep 23 15:33:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:33:51.262292 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f30660-73f9-4ee8-bc0e-dce575ffd022 None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:33:51.272007 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f30660-73f9-4ee8-bc0e-dce575ffd022 None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:33:51.276699 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf0d4ae-9864-440a-a4eb-c43478f2b521 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:33:51.277303 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf0d4ae-9864-440a-a4eb-c43478f2b521 admin admin] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:33:51.277580 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf0d4ae-9864-440a-a4eb-c43478f2b521 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:33:51.277849 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf0d4ae-9864-440a-a4eb-c43478f2b521 admin admin] PATH_INFO: `/v3/projects` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:33:51.278503 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf0d4ae-9864-440a-a4eb-c43478f2b521 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:33:51.280472 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf0d4ae-9864-440a-a4eb-c43478f2b521 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:33:51.366947 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1435/2892] 10.222.0.55 () {60 vars in 1153 bytes} [Fri Sep 23 15:33:51 2022] POST /identity/v3/projects => generated 371 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:33:51.371099 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd4da4a-8d49-44b6-86c5-48e5781fe091 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:33:51.381445 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd4da4a-8d49-44b6-86c5-48e5781fe091 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:33:51.439834 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f734bb5-4dec-4234-b10e-5ca6eb97cb5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:33:51.440499 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f734bb5-4dec-4234-b10e-5ca6eb97cb5a admin admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:33:51.440768 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f734bb5-4dec-4234-b10e-5ca6eb97cb5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:33:51.441044 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f734bb5-4dec-4234-b10e-5ca6eb97cb5a admin admin] PATH_INFO: `/v3/users` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:33:51.441701 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f734bb5-4dec-4234-b10e-5ca6eb97cb5a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:33:51.443770 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f734bb5-4dec-4234-b10e-5ca6eb97cb5a admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:33:51.536939 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1458/2893] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Sep 23 15:33:51 2022] POST /identity/v3/users => generated 286 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:33:51.541660 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf0d4ae-9864-440a-a4eb-c43478f2b521 admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:33:51.552059 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf0d4ae-9864-440a-a4eb-c43478f2b521 admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:33:51.556961 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80b271d-83f7-4573-a947-207239d41884 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:33:51.557615 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80b271d-83f7-4573-a947-207239d41884 admin admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:33:51.557897 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80b271d-83f7-4573-a947-207239d41884 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:33:51.558168 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80b271d-83f7-4573-a947-207239d41884 admin admin] PATH_INFO: `/v3/roles` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:33:51.558812 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80b271d-83f7-4573-a947-207239d41884 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:33:51.560520 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80b271d-83f7-4573-a947-207239d41884 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:33:51.572901 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1436/2894] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Sep 23 15:33:51 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:33:51.577416 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f734bb5-4dec-4234-b10e-5ca6eb97cb5a admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:33:51.588428 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f734bb5-4dec-4234-b10e-5ca6eb97cb5a admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:33:51.593352 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9649932d-a41c-41ec-af04-54efc6e873f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:33:51.593961 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9649932d-a41c-41ec-af04-54efc6e873f0 admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:33:51.594239 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9649932d-a41c-41ec-af04-54efc6e873f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:33:51.594510 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9649932d-a41c-41ec-af04-54efc6e873f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:33:51.595150 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9649932d-a41c-41ec-af04-54efc6e873f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:33:51.596807 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9649932d-a41c-41ec-af04-54efc6e873f0 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:33:51.608142 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1459/2895] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Sep 23 15:33:51 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:33:51.612754 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80b271d-83f7-4573-a947-207239d41884 admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:33:51.623273 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80b271d-83f7-4573-a947-207239d41884 admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:33:51.627908 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38dba81a-7888-4ec8-9ec9-38c0f11c679e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:33:51.628600 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dba81a-7888-4ec8-9ec9-38c0f11c679e admin admin] REQUEST_METHOD: `PUT` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:33:51.628877 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dba81a-7888-4ec8-9ec9-38c0f11c679e admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:33:51.629150 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dba81a-7888-4ec8-9ec9-38c0f11c679e admin admin] PATH_INFO: `/v3/projects/9a369a3ba11e418093f20227e59698db/users/2ae5c14782014e6e9187dee6fd612498/roles/af257be2dee54d6b942f58650196582c` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:33:51.646237 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38dba81a-7888-4ec8-9ec9-38c0f11c679e admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a369a3ba11e418093f20227e59698db, user_id=2ae5c14782014e6e9187dee6fd612498, role_id=af257be2dee54d6b942f58650196582c)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:33:51.649542 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38dba81a-7888-4ec8-9ec9-38c0f11c679e admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:33:51.770489 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1437/2896] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Sep 23 15:33:51 2022] PUT /identity/v3/projects/9a369a3ba11e418093f20227e59698db/users/2ae5c14782014e6e9187dee6fd612498/roles/af257be2dee54d6b942f58650196582c => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:33:51.782633 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734fefaf-c05c-49ed-a236-2b2c3dbcf9aa None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:33:51.782922 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734fefaf-c05c-49ed-a236-2b2c3dbcf9aa None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:33:51.783207 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734fefaf-c05c-49ed-a236-2b2c3dbcf9aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:33:51.812916 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-734fefaf-c05c-49ed-a236-2b2c3dbcf9aa None None] MFA Rules not processed for user `2ae5c14782014e6e9187dee6fd612498`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:33:51.814147 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-734fefaf-c05c-49ed-a236-2b2c3dbcf9aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:33:51.860016 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:33:51.882403 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1460/2897] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:33:51 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 15:33:51.894370 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38dba81a-7888-4ec8-9ec9-38c0f11c679e admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:33:51.896094 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-38dba81a-7888-4ec8-9ec9-38c0f11c679e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:33:51.976400 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38dba81a-7888-4ec8-9ec9-38c0f11c679e admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:33:51.980059 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e0e67d-cb22-4835-a1f2-f1e9d8a60e88 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51c39f1d05634589a88c0b6477bd014e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:33:51.980702 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e0e67d-cb22-4835-a1f2-f1e9d8a60e88 service glance] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:33:51.980996 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e0e67d-cb22-4835-a1f2-f1e9d8a60e88 service glance] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:33:51.981247 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e0e67d-cb22-4835-a1f2-f1e9d8a60e88 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:33:51.991050 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e0e67d-cb22-4835-a1f2-f1e9d8a60e88 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:33:51.993227 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e0e67d-cb22-4835-a1f2-f1e9d8a60e88 service glance] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:33:52.003024 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1438/2898] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:33:51 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:34:01.959045 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=7 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:34:01.998375 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=62,keystone:INSERT=2 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:34:20.773017 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734fefaf-c05c-49ed-a236-2b2c3dbcf9aa None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:34:20.836858 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734fefaf-c05c-49ed-a236-2b2c3dbcf9aa None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:34:20.841446 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886ce00f-0066-40da-861f-b04ca36b0f3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:34:20.842080 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886ce00f-0066-40da-861f-b04ca36b0f3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:34:20.842361 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886ce00f-0066-40da-861f-b04ca36b0f3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:34:20.842685 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886ce00f-0066-40da-861f-b04ca36b0f3e admin admin] PATH_INFO: `/v3/users/2ae5c14782014e6e9187dee6fd612498` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:34:20.843816 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886ce00f-0066-40da-861f-b04ca36b0f3e admin admin] RBAC: Authorizing `identity:delete_user(user_id=2ae5c14782014e6e9187dee6fd612498)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:34:20.845745 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886ce00f-0066-40da-861f-b04ca36b0f3e admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:34:21.018504 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-886ce00f-0066-40da-861f-b04ca36b0f3e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2ae5c14782014e6e9187dee6fd612498'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:34:21.030456 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-886ce00f-0066-40da-861f-b04ca36b0f3e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2ae5c14782014e6e9187dee6fd612498'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:34:21.038014 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-886ce00f-0066-40da-861f-b04ca36b0f3e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2ae5c14782014e6e9187dee6fd612498'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:34:21.169765 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-886ce00f-0066-40da-861f-b04ca36b0f3e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2ae5c14782014e6e9187dee6fd612498'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:34:21.170404 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-886ce00f-0066-40da-861f-b04ca36b0f3e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2ae5c14782014e6e9187dee6fd612498'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:34:21.177147 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1461/2899] 10.222.0.55 () {58 vars in 1224 bytes} [Fri Sep 23 15:34:20 2022] DELETE /identity/v3/users/2ae5c14782014e6e9187dee6fd612498 => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:34:21.185390 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d2fc09-2ae0-4007-abea-8783f07e6c47 None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:34:21.185390 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d2fc09-2ae0-4007-abea-8783f07e6c47 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:34:21.185390 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d2fc09-2ae0-4007-abea-8783f07e6c47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:34:21.202034 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-64d2fc09-2ae0-4007-abea-8783f07e6c47 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:34:21.203388 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-64d2fc09-2ae0-4007-abea-8783f07e6c47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:34:21.264913 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1439/2900] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:34:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:34:21.275205 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886ce00f-0066-40da-861f-b04ca36b0f3e admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:34:21.276696 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-886ce00f-0066-40da-861f-b04ca36b0f3e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:34:21.335209 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886ce00f-0066-40da-861f-b04ca36b0f3e admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:34:21.338800 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e19f74c-adce-4ecb-b977-5d252fe13afd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd259178570864ffeb936ee10dfec6b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:34:21.339408 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e19f74c-adce-4ecb-b977-5d252fe13afd service neutron] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:34:21.339690 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e19f74c-adce-4ecb-b977-5d252fe13afd service neutron] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:34:21.339962 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e19f74c-adce-4ecb-b977-5d252fe13afd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:34:21.350040 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e19f74c-adce-4ecb-b977-5d252fe13afd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:34:21.351751 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e19f74c-adce-4ecb-b977-5d252fe13afd service neutron] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:34:21.361810 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1462/2901] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:34:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:34:21.795161 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d2fc09-2ae0-4007-abea-8783f07e6c47 None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:34:21.796690 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-64d2fc09-2ae0-4007-abea-8783f07e6c47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:34:21.810444 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d2fc09-2ae0-4007-abea-8783f07e6c47 None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:34:21.815078 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a7eb25-4eae-4788-a86b-d51b8000e95d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:34:21.815740 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a7eb25-4eae-4788-a86b-d51b8000e95d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:34:21.816052 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a7eb25-4eae-4788-a86b-d51b8000e95d admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:34:21.816333 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a7eb25-4eae-4788-a86b-d51b8000e95d admin admin] PATH_INFO: `/v3/projects/9a369a3ba11e418093f20227e59698db` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:34:21.817738 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a7eb25-4eae-4788-a86b-d51b8000e95d admin admin] RBAC: Authorizing `identity:delete_project(project_id=9a369a3ba11e418093f20227e59698db)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:34:21.819683 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a7eb25-4eae-4788-a86b-d51b8000e95d admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:34:21.937374 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-84a7eb25-4eae-4788-a86b-d51b8000e95d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9a369a3ba11e418093f20227e59698db', 'request_id': 'req-84a7eb25-4eae-4788-a86b-d51b8000e95d'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:34:21.945481 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-84a7eb25-4eae-4788-a86b-d51b8000e95d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9a369a3ba11e418093f20227e59698db', 'request_id': 'req-84a7eb25-4eae-4788-a86b-d51b8000e95d'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:34:22.038718 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-84a7eb25-4eae-4788-a86b-d51b8000e95d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9a369a3ba11e418093f20227e59698db', 'request_id': 'req-84a7eb25-4eae-4788-a86b-d51b8000e95d'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:34:22.051618 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-84a7eb25-4eae-4788-a86b-d51b8000e95d admin admin] The token cache is being invalidate because project 9a369a3ba11e418093f20227e59698db was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81993) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 15:34:22.052098 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-84a7eb25-4eae-4788-a86b-d51b8000e95d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:34:22.052755 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-84a7eb25-4eae-4788-a86b-d51b8000e95d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:34:22.054233 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1440/2902] 10.222.0.55 () {58 vars in 1233 bytes} [Fri Sep 23 15:34:21 2022] DELETE /identity/v3/projects/9a369a3ba11e418093f20227e59698db => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:34:22.059222 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7e4575-2a4c-4751-8741-1c0a58055d27 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:34:22.059715 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7e4575-2a4c-4751-8741-1c0a58055d27 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:34:22.059715 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7e4575-2a4c-4751-8741-1c0a58055d27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:34:22.080951 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-2d7e4575-2a4c-4751-8741-1c0a58055d27 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:34:22.082999 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-2d7e4575-2a4c-4751-8741-1c0a58055d27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:34:22.099143 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:34:22.141995 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1463/2903] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:34:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:34:22.150233 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce77a6f8-30ea-4353-ad6b-f9d22a9f45b5 None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:34:22.150514 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce77a6f8-30ea-4353-ad6b-f9d22a9f45b5 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:34:22.150786 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce77a6f8-30ea-4353-ad6b-f9d22a9f45b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:34:22.170741 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-ce77a6f8-30ea-4353-ad6b-f9d22a9f45b5 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:34:22.171935 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-ce77a6f8-30ea-4353-ad6b-f9d22a9f45b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:34:22.180323 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1441/2904] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:34:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:34:22.185076 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7e4575-2a4c-4751-8741-1c0a58055d27 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:34:22.195423 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7e4575-2a4c-4751-8741-1c0a58055d27 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:34:22.200569 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814a8a3c-ed01-4eb4-90b1-e5456963bfe3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:34:22.201171 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814a8a3c-ed01-4eb4-90b1-e5456963bfe3 admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:34:22.201452 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814a8a3c-ed01-4eb4-90b1-e5456963bfe3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:34:22.201725 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814a8a3c-ed01-4eb4-90b1-e5456963bfe3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:34:22.202403 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814a8a3c-ed01-4eb4-90b1-e5456963bfe3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81994) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:34:22.202670 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814a8a3c-ed01-4eb4-90b1-e5456963bfe3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:34:22.204280 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814a8a3c-ed01-4eb4-90b1-e5456963bfe3 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:34:22.216132 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1464/2905] 10.222.0.55 () {58 vars in 1153 bytes} [Fri Sep 23 15:34:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 15:34:22.220600 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce77a6f8-30ea-4353-ad6b-f9d22a9f45b5 None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:34:22.229963 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce77a6f8-30ea-4353-ad6b-f9d22a9f45b5 None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:34:22.234623 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569edd8f-5b91-4b37-b97f-0c1f7a09bf86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:34:22.235210 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569edd8f-5b91-4b37-b97f-0c1f7a09bf86 admin admin] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:34:22.235484 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569edd8f-5b91-4b37-b97f-0c1f7a09bf86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:34:22.235778 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569edd8f-5b91-4b37-b97f-0c1f7a09bf86 admin admin] PATH_INFO: `/v3/projects` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:34:22.236722 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569edd8f-5b91-4b37-b97f-0c1f7a09bf86 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:34:22.239215 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569edd8f-5b91-4b37-b97f-0c1f7a09bf86 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:34:22.321013 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1442/2906] 10.222.0.55 () {60 vars in 1153 bytes} [Fri Sep 23 15:34:22 2022] POST /identity/v3/projects => generated 375 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:34:22.325053 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814a8a3c-ed01-4eb4-90b1-e5456963bfe3 admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:34:22.335284 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814a8a3c-ed01-4eb4-90b1-e5456963bfe3 admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:34:22.394682 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa034ff-4ded-4a3a-967f-7453905923bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:34:22.395310 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa034ff-4ded-4a3a-967f-7453905923bc admin admin] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:34:22.395618 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa034ff-4ded-4a3a-967f-7453905923bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:34:22.395890 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa034ff-4ded-4a3a-967f-7453905923bc admin admin] PATH_INFO: `/v3/users` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:34:22.396544 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa034ff-4ded-4a3a-967f-7453905923bc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:34:22.398439 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa034ff-4ded-4a3a-967f-7453905923bc admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:34:22.484591 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1465/2907] 10.222.0.55 () {60 vars in 1144 bytes} [Fri Sep 23 15:34:22 2022] POST /identity/v3/users => generated 288 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 23 15:34:22.489752 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569edd8f-5b91-4b37-b97f-0c1f7a09bf86 admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:34:22.500321 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569edd8f-5b91-4b37-b97f-0c1f7a09bf86 admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:34:22.505347 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98a0e43-4745-4cf0-9a1d-c479e4c6eb54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:34:22.505952 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98a0e43-4745-4cf0-9a1d-c479e4c6eb54 admin admin] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:34:22.506232 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98a0e43-4745-4cf0-9a1d-c479e4c6eb54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:34:22.506507 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98a0e43-4745-4cf0-9a1d-c479e4c6eb54 admin admin] PATH_INFO: `/v3/roles` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:34:22.507147 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98a0e43-4745-4cf0-9a1d-c479e4c6eb54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:34:22.508785 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98a0e43-4745-4cf0-9a1d-c479e4c6eb54 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:34:22.518938 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1443/2908] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Sep 23 15:34:22 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:34:22.524268 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa034ff-4ded-4a3a-967f-7453905923bc admin admin] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:34:22.534994 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa034ff-4ded-4a3a-967f-7453905923bc admin admin] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:34:22.540220 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf6d98c-0602-40ff-bfe8-c2b65d0ca6c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:34:22.540822 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf6d98c-0602-40ff-bfe8-c2b65d0ca6c0 admin admin] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:34:22.541097 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf6d98c-0602-40ff-bfe8-c2b65d0ca6c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:34:22.541366 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf6d98c-0602-40ff-bfe8-c2b65d0ca6c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:34:22.542135 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf6d98c-0602-40ff-bfe8-c2b65d0ca6c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:34:22.543713 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf6d98c-0602-40ff-bfe8-c2b65d0ca6c0 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:34:22.555212 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1466/2909] 10.222.0.55 () {58 vars in 1122 bytes} [Fri Sep 23 15:34:22 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:34:22.559530 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98a0e43-4745-4cf0-9a1d-c479e4c6eb54 admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:34:22.569723 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98a0e43-4745-4cf0-9a1d-c479e4c6eb54 admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:34:22.574395 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08320db1-9040-40f8-8eb3-436e7c1ec629 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:34:22.574993 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08320db1-9040-40f8-8eb3-436e7c1ec629 admin admin] REQUEST_METHOD: `PUT` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:34:22.575265 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08320db1-9040-40f8-8eb3-436e7c1ec629 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:34:22.575545 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08320db1-9040-40f8-8eb3-436e7c1ec629 admin admin] PATH_INFO: `/v3/projects/e350c92d9bb44abe998c6f28c20a6356/users/e46d3e40d2cd4653a0b997845dccbeff/roles/af257be2dee54d6b942f58650196582c` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:34:22.590123 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08320db1-9040-40f8-8eb3-436e7c1ec629 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e350c92d9bb44abe998c6f28c20a6356, user_id=e46d3e40d2cd4653a0b997845dccbeff, role_id=af257be2dee54d6b942f58650196582c)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:34:22.592799 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08320db1-9040-40f8-8eb3-436e7c1ec629 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:34:22.668503 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1444/2910] 10.222.0.55 () {60 vars in 1483 bytes} [Fri Sep 23 15:34:22 2022] PUT /identity/v3/projects/e350c92d9bb44abe998c6f28c20a6356/users/e46d3e40d2cd4653a0b997845dccbeff/roles/af257be2dee54d6b942f58650196582c => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:34:22.679143 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b81a6c4-df4e-4d5f-ad81-3f46338fa294 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:34:22.679435 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b81a6c4-df4e-4d5f-ad81-3f46338fa294 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:34:22.679713 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b81a6c4-df4e-4d5f-ad81-3f46338fa294 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:34:22.708562 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-6b81a6c4-df4e-4d5f-ad81-3f46338fa294 None None] MFA Rules not processed for user `e46d3e40d2cd4653a0b997845dccbeff`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:34:22.709792 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-6b81a6c4-df4e-4d5f-ad81-3f46338fa294 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:34:22.774459 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1467/2911] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:34:22 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:34:22.785258 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08320db1-9040-40f8-8eb3-436e7c1ec629 admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:34:22.787763 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-08320db1-9040-40f8-8eb3-436e7c1ec629 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:34:22.809656 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:34:22.845355 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08320db1-9040-40f8-8eb3-436e7c1ec629 admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:34:22.849715 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4986bf-b44d-4855-906d-73750b1f06cc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feca6278be904ef8b0dc580a67e477f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:34:22.850324 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4986bf-b44d-4855-906d-73750b1f06cc service cinder] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:34:22.850607 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4986bf-b44d-4855-906d-73750b1f06cc service cinder] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:34:22.850884 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4986bf-b44d-4855-906d-73750b1f06cc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:34:22.859370 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4986bf-b44d-4855-906d-73750b1f06cc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:34:22.860832 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4986bf-b44d-4855-906d-73750b1f06cc service cinder] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:34:22.870823 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1445/2912] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:34:22 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:34:24.590988 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76816bf1-d5f6-499c-9498-234c5721d634 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:34:24.591532 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76816bf1-d5f6-499c-9498-234c5721d634 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:34:24.591532 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76816bf1-d5f6-499c-9498-234c5721d634 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:34:24.610445 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-76816bf1-d5f6-499c-9498-234c5721d634 None None] MFA Rules not processed for user `e46d3e40d2cd4653a0b997845dccbeff`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:34:24.611898 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-76816bf1-d5f6-499c-9498-234c5721d634 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:34:24.618894 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1468/2913] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:34:24 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:34:32.935726 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=17 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:34:34.605005 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=92,keystone:INSERT=3 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:34:42.398816 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d661d29d-865e-4e0b-a47d-a978aea17f9e None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:34:42.399304 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d661d29d-865e-4e0b-a47d-a978aea17f9e None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:34:42.399518 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d661d29d-865e-4e0b-a47d-a978aea17f9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:34:42.451831 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-d661d29d-865e-4e0b-a47d-a978aea17f9e None None] MFA Rules not processed for user `68206a6b4f244d45bf7bf1504396e430`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:34:42.453092 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-d661d29d-865e-4e0b-a47d-a978aea17f9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:34:42.521131 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1446/2914] 10.222.0.55 () {58 vars in 1009 bytes} [Fri Sep 23 15:34:42 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:34:42.711900 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76816bf1-d5f6-499c-9498-234c5721d634 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:34:42.713342 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-76816bf1-d5f6-499c-9498-234c5721d634 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:34:42.806796 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76816bf1-d5f6-499c-9498-234c5721d634 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:34:42.811867 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e530df78-f736-4670-badc-fb97dd720dbf service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2228c0b49a0249bbaf524a8c4e9e108b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:34:42.812508 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e530df78-f736-4670-badc-fb97dd720dbf service swift] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:34:42.812795 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e530df78-f736-4670-badc-fb97dd720dbf service swift] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:34:42.813073 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e530df78-f736-4670-badc-fb97dd720dbf service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:34:42.822149 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e530df78-f736-4670-badc-fb97dd720dbf service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:34:42.823729 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e530df78-f736-4670-badc-fb97dd720dbf service swift] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:34:42.831838 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1469/2915] 10.222.0.55 () {60 vars in 1384 bytes} [Fri Sep 23 15:34:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 23 15:34:43.618324 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8b4cb5-74c9-44eb-a01e-79751ba8d967 None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:34:43.618824 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8b4cb5-74c9-44eb-a01e-79751ba8d967 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:34:43.618943 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8b4cb5-74c9-44eb-a01e-79751ba8d967 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:34:43.639693 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-6f8b4cb5-74c9-44eb-a01e-79751ba8d967 None None] MFA Rules not processed for user `68206a6b4f244d45bf7bf1504396e430`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:34:43.640931 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-6f8b4cb5-74c9-44eb-a01e-79751ba8d967 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:34:43.648505 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1447/2916] 10.222.0.55 () {58 vars in 967 bytes} [Fri Sep 23 15:34:43 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:34:43.660785 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e530df78-f736-4670-badc-fb97dd720dbf service swift] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:34:43.671994 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e530df78-f736-4670-badc-fb97dd720dbf service swift] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:34:43.676993 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9884560-1842-4405-903b-36d81564385a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2228c0b49a0249bbaf524a8c4e9e108b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:34:43.677612 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9884560-1842-4405-903b-36d81564385a service swift] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:34:43.677880 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9884560-1842-4405-903b-36d81564385a service swift] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:34:43.678158 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9884560-1842-4405-903b-36d81564385a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:34:43.686530 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9884560-1842-4405-903b-36d81564385a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:34:43.688140 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9884560-1842-4405-903b-36d81564385a service swift] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:34:43.696384 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1470/2917] 10.222.0.55 () {60 vars in 1384 bytes} [Fri Sep 23 15:34:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 23 15:34:44.291054 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adf57d8-d44e-447f-aab5-d15f0761f5e5 None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:34:44.291461 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adf57d8-d44e-447f-aab5-d15f0761f5e5 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:34:44.291827 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adf57d8-d44e-447f-aab5-d15f0761f5e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:34:44.312235 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-6adf57d8-d44e-447f-aab5-d15f0761f5e5 None None] MFA Rules not processed for user `68206a6b4f244d45bf7bf1504396e430`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:34:44.313521 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-6adf57d8-d44e-447f-aab5-d15f0761f5e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:34:44.321799 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1448/2918] 10.222.0.55 () {58 vars in 1009 bytes} [Fri Sep 23 15:34:44 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:34:44.334220 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9884560-1842-4405-903b-36d81564385a service swift] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:34:44.344849 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9884560-1842-4405-903b-36d81564385a service swift] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:34:44.350247 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24466828-f679-40ca-8808-cee8d2496ca7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2228c0b49a0249bbaf524a8c4e9e108b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:34:44.350852 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24466828-f679-40ca-8808-cee8d2496ca7 service swift] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:34:44.351131 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24466828-f679-40ca-8808-cee8d2496ca7 service swift] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:34:44.351486 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24466828-f679-40ca-8808-cee8d2496ca7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:34:44.359966 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24466828-f679-40ca-8808-cee8d2496ca7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:34:44.361898 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24466828-f679-40ca-8808-cee8d2496ca7 service swift] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:34:44.370673 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1471/2919] 10.222.0.55 () {60 vars in 1384 bytes} [Fri Sep 23 15:34:44 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 23 15:34:44.938925 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8c746b-f970-4bd1-a70f-f9bbdbbdd421 None None] REQUEST_METHOD: `POST` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:34:44.939417 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8c746b-f970-4bd1-a70f-f9bbdbbdd421 None None] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:34:44.939530 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8c746b-f970-4bd1-a70f-f9bbdbbdd421 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:34:44.958356 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.auth.core [None req-db8c746b-f970-4bd1-a70f-f9bbdbbdd421 None None] MFA Rules not processed for user `68206a6b4f244d45bf7bf1504396e430`. Rule list: `[]` (Enabled: `True`). {{(pid=81993) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:34:44.959532 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-db8c746b-f970-4bd1-a70f-f9bbdbbdd421 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:34:44.966993 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1449/2920] 10.222.0.55 () {58 vars in 967 bytes} [Fri Sep 23 15:34:44 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:34:44.979464 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24466828-f679-40ca-8808-cee8d2496ca7 service swift] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:34:44.990242 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24466828-f679-40ca-8808-cee8d2496ca7 service swift] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:34:44.996091 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfff193-caaf-4703-ba0f-630f2bd40203 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2228c0b49a0249bbaf524a8c4e9e108b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:34:44.996693 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfff193-caaf-4703-ba0f-630f2bd40203 service swift] REQUEST_METHOD: `GET` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:34:44.996979 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfff193-caaf-4703-ba0f-630f2bd40203 service swift] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:34:44.997254 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfff193-caaf-4703-ba0f-630f2bd40203 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:34:45.005489 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfff193-caaf-4703-ba0f-630f2bd40203 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:34:45.007109 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfff193-caaf-4703-ba0f-630f2bd40203 service swift] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:34:45.015332 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1472/2921] 10.222.0.55 () {60 vars in 1384 bytes} [Fri Sep 23 15:34:44 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 23 15:34:47.690442 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8c746b-f970-4bd1-a70f-f9bbdbbdd421 None None] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:34:47.753378 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8c746b-f970-4bd1-a70f-f9bbdbbdd421 None None] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:34:47.758252 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71368f79-aab0-4e8a-8ac7-a0288bff8ba9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:34:47.758894 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71368f79-aab0-4e8a-8ac7-a0288bff8ba9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:34:47.759177 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71368f79-aab0-4e8a-8ac7-a0288bff8ba9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:34:47.759481 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71368f79-aab0-4e8a-8ac7-a0288bff8ba9 admin admin] PATH_INFO: `/v3/users/e46d3e40d2cd4653a0b997845dccbeff` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:34:47.760656 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71368f79-aab0-4e8a-8ac7-a0288bff8ba9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e46d3e40d2cd4653a0b997845dccbeff)` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:34:47.762657 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71368f79-aab0-4e8a-8ac7-a0288bff8ba9 admin admin] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:34:47.923327 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-71368f79-aab0-4e8a-8ac7-a0288bff8ba9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e46d3e40d2cd4653a0b997845dccbeff'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:34:47.928776 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:34:47.936702 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-71368f79-aab0-4e8a-8ac7-a0288bff8ba9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e46d3e40d2cd4653a0b997845dccbeff'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:34:47.943556 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-71368f79-aab0-4e8a-8ac7-a0288bff8ba9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e46d3e40d2cd4653a0b997845dccbeff'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:34:48.009707 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-71368f79-aab0-4e8a-8ac7-a0288bff8ba9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e46d3e40d2cd4653a0b997845dccbeff'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:34:48.010310 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.notifications [None req-71368f79-aab0-4e8a-8ac7-a0288bff8ba9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e46d3e40d2cd4653a0b997845dccbeff'} {{(pid=81993) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:34:48.016603 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1450/2922] 10.222.0.55 () {58 vars in 1224 bytes} [Fri Sep 23 15:34:47 2022] DELETE /identity/v3/users/e46d3e40d2cd4653a0b997845dccbeff => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:34:48.021214 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f31342-71d9-4364-8b23-2e140ab7d993 None None] REQUEST_METHOD: `POST` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:34:48.021728 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f31342-71d9-4364-8b23-2e140ab7d993 None None] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:34:48.021873 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f31342-71d9-4364-8b23-2e140ab7d993 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:34:48.042243 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.auth.core [None req-12f31342-71d9-4364-8b23-2e140ab7d993 None None] MFA Rules not processed for user `0fc10e0211a54a0faf7ea9b86542de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81994) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:34:48.044178 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-12f31342-71d9-4364-8b23-2e140ab7d993 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:34:48.104835 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1473/2923] 10.222.0.55 () {58 vars in 958 bytes} [Fri Sep 23 15:34:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:34:48.114654 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71368f79-aab0-4e8a-8ac7-a0288bff8ba9 admin admin] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:34:48.116087 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-71368f79-aab0-4e8a-8ac7-a0288bff8ba9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:34:48.177024 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71368f79-aab0-4e8a-8ac7-a0288bff8ba9 admin admin] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:34:48.180804 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc5df9e-e7fe-46e9-bad3-e09cc571afd0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd259178570864ffeb936ee10dfec6b28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:34:48.181406 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc5df9e-e7fe-46e9-bad3-e09cc571afd0 service neutron] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:34:48.181683 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc5df9e-e7fe-46e9-bad3-e09cc571afd0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:34:48.181949 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc5df9e-e7fe-46e9-bad3-e09cc571afd0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:34:48.191125 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc5df9e-e7fe-46e9-bad3-e09cc571afd0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:34:48.192682 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc5df9e-e7fe-46e9-bad3-e09cc571afd0 service neutron] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:34:48.202454 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1451/2924] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:34:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:34:48.422927 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f31342-71d9-4364-8b23-2e140ab7d993 None None] Authenticating user token {{(pid=81994) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:34:48.428665 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.fernet_utils [None req-12f31342-71d9-4364-8b23-2e140ab7d993 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81994) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:34:48.440843 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f31342-71d9-4364-8b23-2e140ab7d993 None None] Validating token access rules against request {{(pid=81994) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:34:48.446469 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31550a14-2bed-46ad-962b-a0e12fc0fd98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc10e0211a54a0faf7ea9b86542de03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23b5c58d198d4456bfeec0a8daae1113', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81994) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:34:48.447173 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31550a14-2bed-46ad-962b-a0e12fc0fd98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:34:48.448101 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31550a14-2bed-46ad-962b-a0e12fc0fd98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:34:48.451665 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31550a14-2bed-46ad-962b-a0e12fc0fd98 admin admin] PATH_INFO: `/v3/projects/e350c92d9bb44abe998c6f28c20a6356` {{(pid=81994) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:34:48.451665 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31550a14-2bed-46ad-962b-a0e12fc0fd98 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e350c92d9bb44abe998c6f28c20a6356)` {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:34:48.451665 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31550a14-2bed-46ad-962b-a0e12fc0fd98 admin admin] RBAC: Authorization granted {{(pid=81994) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:34:48.520090 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:34:48.537352 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-31550a14-2bed-46ad-962b-a0e12fc0fd98 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e350c92d9bb44abe998c6f28c20a6356', 'request_id': 'req-31550a14-2bed-46ad-962b-a0e12fc0fd98'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:34:48.543569 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-31550a14-2bed-46ad-962b-a0e12fc0fd98 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e350c92d9bb44abe998c6f28c20a6356', 'request_id': 'req-31550a14-2bed-46ad-962b-a0e12fc0fd98'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:34:48.549490 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-31550a14-2bed-46ad-962b-a0e12fc0fd98 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e350c92d9bb44abe998c6f28c20a6356', 'request_id': 'req-31550a14-2bed-46ad-962b-a0e12fc0fd98'} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:34:48.615629 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-31550a14-2bed-46ad-962b-a0e12fc0fd98 admin admin] The token cache is being invalidate because project e350c92d9bb44abe998c6f28c20a6356 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81994) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 15:34:48.615951 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-31550a14-2bed-46ad-962b-a0e12fc0fd98 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:34:48.616741 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG keystone.notifications [None req-31550a14-2bed-46ad-962b-a0e12fc0fd98 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81994) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:34:48.618248 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: [pid: 81994|app: 0|req: 1474/2925] 10.222.0.55 () {58 vars in 1233 bytes} [Fri Sep 23 15:34:48 2022] DELETE /identity/v3/projects/e350c92d9bb44abe998c6f28c20a6356 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:34:58.198210 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=36,keystone:DELETE=2,keystone:INSERT=1 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:34:58.621212 dv-cinder-855307-2-iscsi devstack@keystone.service[81994]: DEBUG dbcounter [-] [81994] Writing DB stats keystone:DELETE=6,keystone:SELECT=14,keystone:INSERT=1 {{(pid=81994) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:38:13.678745 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc5df9e-e7fe-46e9-bad3-e09cc571afd0 service neutron] Authenticating user token {{(pid=81993) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:38:13.680272 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-7fc5df9e-e7fe-46e9-bad3-e09cc571afd0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:38:13.830993 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc5df9e-e7fe-46e9-bad3-e09cc571afd0 service neutron] Validating token access rules against request {{(pid=81993) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:38:13.892992 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5d89b7-d8c0-480d-98d4-720f0a20af64 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09f1a55d180642c781a6e0b017d9f0aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b3e9d4b6474bd4a04ba59ef52bf3e2', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81993) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:38:13.893654 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5d89b7-d8c0-480d-98d4-720f0a20af64 service placement] REQUEST_METHOD: `GET` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:38:13.893961 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5d89b7-d8c0-480d-98d4-720f0a20af64 service placement] SCRIPT_NAME: `/identity` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:38:13.894295 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5d89b7-d8c0-480d-98d4-720f0a20af64 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81993) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:38:13.896030 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.fernet_utils [None req-6f5d89b7-d8c0-480d-98d4-720f0a20af64 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81993) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:38:13.907602 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5d89b7-d8c0-480d-98d4-720f0a20af64 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:38:13.909214 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5d89b7-d8c0-480d-98d4-720f0a20af64 service placement] RBAC: Authorization granted {{(pid=81993) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:38:13.919760 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: [pid: 81993|app: 0|req: 1452/2926] 10.222.0.55 () {60 vars in 1365 bytes} [Fri Sep 23 15:38:13 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 242 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:38:23.915431 dv-cinder-855307-2-iscsi devstack@keystone.service[81993]: DEBUG dbcounter [-] [81993] Writing DB stats keystone:SELECT=89 {{(pid=81993) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}